May. 13th, 2013

go_gentle: (mathematician)
I told twitter about this already, but, argh.

So, this weekend I was was working on a text-munging exercise, basically, just for shits and giggles. And it so happened that the text in question was a grab of a bunch of data off of twitter's API, and the exercise asked you to generate certain facts about your dataset. So I blew through this in about 2.5 hours of coding fugue, because what the API gives you is just JSON, and the exercise was in Python, and JSON in Python is just dicts, and reading through and re-arranging dicts is basically the only thing I'm good at in Python.

And then I went and took a shower, and I was thinking in the shower - dangerous, I know - but one of the things that the exercise asked us to do was to identify the sentiment of tweets - how strongly positive or negative the tweet was, based on the presence of various pre-scored emotion words, and then later it asked us to pull out the location data of tweets, and decide which state had the happiest tweets on average, and things like that.

So there I was, in the shower, thinking, and what I thought was, "man, I bet you could cue up the API grab to grab a chunk of tweets off a specific hashtag, and you could toss a sleep in there, so you just grab a few hundred tweets every minute, and you could do it over the course of a hockey game or something, and then you could parse out the sentiment by location, and see how different cities felt about the game." and then I kept thinking, about a graph one of my co-workers showed me several weeks ago, where she had graphed density of a certain metric over a map of North America, in R (which I allegedly know), so now it's not just a table of scores, but a map showing the feelings, in color. And then I said to myself, I bet it's not that hard to make GIFs from a bunch of still graphs - you could automate the whole thing to generate graphs by timestamp, then GIF them together, and you'd be able to see when Toronto gets excited, or when Boston gets sad - little clusters of red and green popping up over North America as goals get scored.

"It's a bummer," I said to myself, "that I'd need to pull the data live tomorrow evening - I don't have time tonight to figure out how to handle the repeated API call yet, and I bet Game 7 would cause a lot of feelings. Oh well. There'll be other games that have just as many emotions, surely."


go_gentle: (Default)
just a girl who's afraid of the dark

April 2015

5 67891011

Style Credit

Expand Cut Tags

No cut tags