Showing posts with label engineering. Show all posts
Showing posts with label engineering. Show all posts

Sunday, November 22, 2015

Slackbot with AWS Lambda // Bourbon Blueberry Shrub

Slackbot with AWS Lambda // Bourbon Blackberry Shrub

If you're looking for the booze to give yourself a Ballmer peak, skip below...

The Code



Slack offers a ton of really neat integrations.  However, in the setting where I use it, most of them aren't an option.  Instead, I recently attended a coding night and spent some time rolling my own simple Slackbot using Amazon Web Services' Lambda and API Gateway features: Slackbroker!

Slackbroker is triggered using a slash command followed by a list of stock symbols.  It looks up the current stock price (or the closing price if the markets are closed) of each symbol, then posts publicly to the room where the slash command was run in the first place reporting the price.  There are a few barriers here thanks to the way slash commands work in Slack:
  • Slash commands call a URL with a payload, but AWS Lambda needs to be invoked by its ARN
  • The payload from a slash command is delivered in URL format (contrary to Slack documentation), but AWS Lambda demands input in JSON
  • Slash commands respond by displaying a private message to the user who invoked them, but I wanted an individual to bring the whole group's attention to a stock

The Business Logic

To be frank, I finished this part last, but for the tutorial's sake I'll explain the entire Lambda contents here instead.  Lambda is pretty easy to set up - by default it accepts your code in its web editor.  While I can't fully get behind it for doing something significant with Lambda, it makes it convenient for small hacks like this.

It's in Python. lambda_handler is a special function that serves as the entry point executed when someone invokes your Lambda's ARN.  Because of the way the URL data is converted later, we get our content in a dictionary inside of event (also a dictionary) at entry postBody, which maps fields designated in the Slack documentation to their values.  

text is a string with everything following the slash command prompt typed in Slack; that is, if a user types /ticker abc def, text looks like 'abc def'.

channel_name is just that - the name of the channel the user was in when they invoked the slash command.  This is helpful to ensure you don't spam a general channel when you have Slackbot post back for you.

This weird URL is actually just a Slack incoming webhook.  They're pretty trivial to set up, so I won't really get into it, but this is the icky bit that lets Slackbroker post back to the entire group, instead of replying in a hidden message.

get_quote is a simple web scraper and is mostly stolen from a great StackOverflow post, but that code references an automatically-named tag in the regular expression - that is, that code only works for one symbol (and the tag name needs to be manually determined for that symbol to work).  I found that there was also a human-readable tag that contains the price, and referenced that in my regex instead.

pretty_quotes is a neat one-liner that uses join and a foreach to turn an array of strings into a single cute comma-separated list for human consumption.  I admit to lack of Python fu; an old classmate gave me a lot of help with this line.

Calling Lambda via URL

Luckily, AWS has a product called API Gateway that solves exactly our problem - it exposes (ugly) URLs that, when called, deliver the payload on to a specified ARN.  Here is how to make a new API Gateway and point it at your Lambda:

  1. Create a new API through API Gateway.  Name it whatever, nobody will see it but you.
  2. It looks pretty empty, so Create Method.  We want a POST with integration type Lambda Function.  Pick the region where your Lambda lives (this is found in the top right corner of the AWS console) and enter the name of your new Lambda.
Now you have a nice-looking flow diagram!  But it's not active.  Make sure you Deploy API, creating a new stage if you didn't have one.

Converting URL Data to JSON

We have a problem, though.  Slack will still send its data in a URL-encoded string, but Lambda expects JSON for it to turn into a Python dictionary!  Luckily, someone thought of this, and API Gateway allows you to write a converter between types.  Head to the Integration Request block of your API, expand Mapping Templates, and add a new one.  Content-Type refers to the type of the incoming data; in our case it's application/x-www-form-urlencoded.  Change the mode from Input Passthrough to Mapping Template; you will be presented with an editor to write your template.  We chose a pretty simple one; $input is a special value in the template.

{
  "postBody" : $input.json("$")
}

This defines what the event object from above will look like - we get an entry postBody that contains a map of all of our arguments from Slack.  Great, now we can actually invoke our Lambda correctly without it complaining about receiving literally anything besides JSON! (It's picky, isn't it?)

Adding Your Slash Command to Slack

We finally have everything ready to set up the Slack side of things!  To get ready, from your API Gateway console, you can use the Resources dropdown as shown here to select Stages; once you do so, choose the stage you created earlier.  Grab the Invoke URL from here.  



Then, head to my-team.slack.com/services/new and add a new Slash Command (way down in the DIY section).  Name it, and in the URL field, paste that Invoke URL from your API Gateway.  Leave the method as POST and customize the rest of the form as you see fit; save it and give it a whirl!

After following all these steps, you should have a working Slackbroker.  If I missed something, please let me know in the comments.  Good luck!

And hey -- do you need a drink after all that?  You're in luck, because so did I....

The Booze



A friend of mine started telling me about shrubs, these fruity vinegar syrups.  I tried making my own and it was none too pretty - somehow whatever I had created just made me feel sick every time I drank it!  But it tasted great, and my interest in shrubs didn't wane - I just decided to leave it to the professionals.  Recently I tried a blueberry lavender one at Finger Lakes Cider House in Ithaca, NY and it had to come home; I discovered it pairs great with bourbon!

Bourbon Blueberry Shrub

1.5 oz bourbon
.75 oz Good Life Cider blueberry-lavender shrub
club soda
ice
Fill a tumbler with ice. (Mine's a chilly Death Star!)  Add the bourbon and shrub; top with club soda and stir.  Condescendingly explain to all your friends what a shrub is and enjoy.

Friday, February 15, 2013

sscanf() Gotcha

Interesting gotcha I ran across using sscanf() today.

Here's the example scenario.  In my case, I was parsing a .mcs file and looking for pairs of characters that were hex representations of bytes of data.  That is, the string "FF001122" would translate to data = {0xFF, 0x00, 0x11, 0x22}.


fstream stream;
stream.open(fileLoc);

char data[8] = {};
char line[16];
stream.getline( line, 16 );

for (int i=0; i<8; i++) {
    sscanf( line + (i*2), "%2x", &(data[i]) );
}


Upon program exit, I get a stack corruption error around 'data'. Weird, right? We don't go out of bounds at any point on data, right?

Well, we don't... But sscanf() is a different story.

Let's step through the for loop.  We'll say line = "0011223344556677".

i=0.
data = { ?, ?, ?, ?, ?, ?, ?, ?}
Perform the sscanf.
data = { 00, 00, 00, 00, ?, ?, ?, ? }

See the problem yet?
We'll keep going:

i=1.
data = {00, 00, 00, 00, ?, ?, ?, ?}
Perform the sscanf.
data = {00, 11, 00, 00, 00, ?, ?, ?}

At the end, we have:

i=7.
data = {00, 11, 22, 33, 44, 55, 66, 77} 00, 00, 00 (out of bounds!!!)

What's going on here is that sscanf() doesn't realize that data is an array of chars.  It assumes that since the "%x" identifier was used, we want it to give us unsigned ints.  But it doesn't check, or ask.  Turns out there's a way to specify manually - the correct form to receive data back in char size (and thus write out of bounds) is to use the identifier string "%2hhx" - that is, 2 hex chars output into one byte of data (specified by the "hh").  If you look carefully, the scanf formatting table in the spec tells you that this is expected behavior, but it's not particularly outspoken about it.

So, keep an eye out, from your good friend Emily, wasting four hours of her day so you don't have to.

EDIT:
Turns out this doesn't quite work depending on your architecture and whether or not you're using Unicode.... I ran this same code in Unicode and the 'hh' format string doesn't help - sscanf still reads into 4 bytes every time.  Internet searches show that the best solution is to use a temporary variable. Barf.

Sunday, May 27, 2012

Automatic Android App Builds with Github, Travis and Ant

Continuous integration is awesome.  Android is okay, I guess.  Both together is pretty darn cool.

If you're using a version control system, which you should be, you may be interested in automatically building your project every time you push a change to your repository.  You can do this in a few ways - traditionally, you can host a Hudson or Jenkins instance on your local machine and set up a commit hook that tells your CI instance that it's time to go make a build.  Your CI bot will then go fetch the source from your new commit and build it, then report the results to you.  However, then the build is still using your PC's resources, so there's not that much benefit (especially if someone somewhere else pushes to your repository with the correct commit hook unexpectedly).  If you have your own server somewhere, you can use that, problem solved...

Not all of us do, though - and for that there are a few sites that will host for you.  One is CloudBees (which I have not used and will not talk about), and the other is Travis, which I did use, and will talk about.

Travis is tightly integrated with GitHub, and doesn't expose a buildbot like Hudson or Jenkins.  It doesn't expose its service hooks, either, so as far as I can tell, it doesn't support hooking into repositories that aren't hosted on GitHub - if you want to use it, you need to mirror your source to GitHub.

That said, it is incredibly easy to use with GitHub.  Simply sign in with your GitHub credentials on the Travis website - in your account properties you can "switch on" projects that you want Travis to build on commit.

Travis uses a single descriptor YAML file named .travis.yml in the root directory of your repository to execute the build.  Your .travis.yml defines which language Travis should use to build, where, and which pre- and post-build steps to take.  A list of supported languages can be found on the Travis documentation here.

In the case of Android projects, Travis does not come with the Android SDK, so you need to download the SDK and the correct Android release libraries yourself in the prebuild by specifying a before_script.

before_script:
    - sudo wget -q http://dl.google.com/android/android-sdk_r18-linux.tgz
    - sudo tar xzvf android-sdk_r18-linux.tgz > /dev/null
    - sudo android-sdk-linux/tools/android -s update sdk --filter android-7,platform-tool --no-ui
    - sudo android-sdk-linux/tools/android -s update project --path ./FridgeMagnet --target "android-7"
    - sudo android-sdk-linux/tools/android -s update test-project --path ./FridgeMagnetTest -m ../FridgeMagnet
I'm building my project based on Android 2.1, so I am downloading android-7 (Android 2.1).  android update takes an ID to determine which packages to download - you can see a list of the correct IDs for each available package using android list sdk and specify which packages to download using the --filter argument.  platform-tool is needed to build with ant.  android update project sets up the ant build.xml for each project - needed for the next step. In some cases,  you may find that the permissions in your Travis environment are limiting, so I use a big hammer and simply add:
- sudo chmod -R 777 ./*
at the bottom of my before_script.

Travis downloads source to a non-persistent environment with every push, so it is necessary to download the SDKs every time.  This is the suggested method.

Next we want to actually build, using ant.  You can specifiy your build script under script: in your .travis.yml.  In the most simple case you can simply specify:
script:
- cd ./FridgeMagnet
- ant clean
- ant debug
If you have more than one project build, you can use a bash script as your script: action instead of specifying each action in the .travis.yml - otherwise if one project build fails, Travis will stop the build process completely.  You can then return the success status from your script so Travis knows whether your build ran correctly or not.
#!/bin/bash
FAILED=0
cd ./FridgeMagnet
ant clean
ant debug
if [ "$?" = 1 ]; then
echo "FridgeMagnet build failed!"
    FAILED=1
fi
cd ..
exit $FAILED
Now you should be able to see your builds on Travis as they are built under http://travis-ci.org/#!/githubusername/GitHubProjectName.

For an example, my project FridgeMagnet can be found on GitHub and Travis.

Wednesday, November 2, 2011

Feminine Wiles

I'll be outright.

Every time I want to make friends with males in my major, I feel pressured to prove myself.  I feel pressured to make sure they know I work for my own success and am capable in my field.  I don't feel that I will be respected without making it known I excel in my field.

I'm sick of this.  I'm sick of feeling like every new friend is another point to prove, and every first impression is against me.  I strive to present myself in a way that first impressions of me are positive and respectful, and I have infinite doubts as to this work's payoff.  I still feel as though I have a number of stereotypes to overcome: unmotivated, unwilling to learn, airheaded, flippant, and more.  It sucks fighting against those stereotypes.

(A quick aside, as an exercise to the tech-student reader.  Ask yourself, "Do I feel that women in tech fields are any of those things?"  Now, disregard your politically-correct heavily-generalized answer and ask yourself, "What do I think of the women in my tech classes?"  Ponder on that for a moment.)

But it sucks even more that those stereotypes are there in the first place.  It sucks that there were enough women in technical fields before me who fit those traits to form a stereotype.  And of course, it sucks that the behavior of those women should have any bearing on me.

I'm interested in talking about why women have stereotypes like this.  I think it has to do with these so-called "feminine wiles" - a shift of the blouse and a wink of the eye, and a good-looking woman has the world at her feet, so we're told.  And it works, often enough, for the same reason that a glance-over and a cute smile can get plenty of men whatever they want with us - humans like feeling wanted, but that is neither here nor there.  Using these feminine wiles, to me, seems akin to using a CAS calculator on the ACT.  Yes, you can probably get away with it.  Yes, it will probably work out well for you.  But morality aside, you won't learn anything and you won't represent yourself accurately.  You aren't showing any real skills.  And if you're caught, you're up shit creek.

Men are easy, in some fields more than others.  The fields themselves are not!  Sure, you can get help with your homework and breeze through freshman year with the assistance of your reverse-harem.  But what will that do for you?  It will ensure that you have a shaky background for the next year, and the year after that, ad nauseum.  And it will build you a reputation, which will in turn build your gender-specific peers a reputation.  Your easy way through engineering school is leaving behind red shells for everyone who comes after you.  It's a dangerous game to play, and most of the time it'll bite you in the ass - imagine, years later, you apply for a job at Company Foo.  The recruiter at Foo says to student Bob, who he knows went to school with you, "What do you think of this person who we are thinking of hiring?"  If Bob says, "Well, she didn't do much work in college - usually she had a boyfriend do her work for her," this is a problem.  This is a reputation that you will not be able to get rid of, even if you fix your experience and your skill base after graduating!

My plea here has a dual nature.  Women, and men, everyone: please think long and hard about the way you present yourself to the world, and about any and all effects your demeanor has on what people think of "your kind".  And women, men, everyone: please, if you catch yourself making a snap judgement of someone and they fit a negative stereotype - take notice and ask yourself what you have observed to allow this judgement.  Please don't fear the benefit of the doubt - let your peers prove themselves guilty if they so are.  It's much easier than proving oneself innocent.

There are more "wiles" than this blatant example, though.  A wise piece of advice was once offered to me - the giver admonished that instead of striving to prove myself to every male I meet, I should instead stay passive and agreeable.  She suggested that if I compete to prove a point with every male, most will insist on winning for their pride's sake, and would continue competing until they win.  She suggested that I essentially lay low and do my own thing unobtrusively.  And yeah, that sounds like a pretty effective strategy - but a cowardly one.  I'm young and I'm outspoken and I want to make it known that I know what I'm doing.  I don't want to hide my talent because of my chromosomes.  I don't see a reason that I should have to.  Some men have competitive natures, some have unobtrusive natures, so why should I hide my competitive nature if I have one?  Please, please - give me a good reason.

I have a deeper concern, though, than my own selfish concern - and that is that hiding talent could easily be transposed into letting talent go unused.  It is absolutely true that if you don't use it, you lose it.  So what does that say about the long-term benefits of women hiding their intelligence, interests, assertiveness, and talent?  By being unobtrusive, are we allowing society to perpetuate our own downfall?  The statistics already aren't in our favor, so why shouldn't we do anything and everything in our power to keep them from getting worse?