Remote Beta Students Analyze Twitter Data Using Watson

A team of students in our Remote program used Watson, IBM’s natural language processor, in concert with the Twitter API to build a library that developers can use for language analysis and translation. Personify, built by Essam Al Joubori, Rohan Agrawal and Phil Elauria, streamlines the process of building apps that work with Watson’s language analysis and translation abilities and Twitter’s APIs. The project immediately garnered a lot of attention, making the front page of Hacker News, and receiving over 150 stars on GitHub.

Watson, best known for winning Jeopardy in 2011, is an Artificial Intelligence system focused on one of computing’s most intractable problems: natural language processing. Recently, Watson opened up certain functionalities for developers, and the team wanted to use these in a project. After examining the various available capabilities of Watson, they decided to focus on translation and a process of personality trait analysis called “user modeling”.

The team demoed Personify at our most recent Hiring Day:

“Putting together Personify was about figuring out what we can do with the features we took from Twitter and the features we took from Watson,” Elauria describes. “It was a mixture of utilizing the user modeling features or the translation features and filtering them down through Twitter.”

To show what Personify is capable of, the trio built a sample app that shows personality traits of Twitter users based on location and keyword. For instance, one could see how people using the words “Super Bowl” in California compare on a personality level to the United States as a whole. This app used Watson’s user modeling feature, which spots certain words associated with established personality types.

“User modeling takes advantage of something called Big Five,” says Elauria. “Big Five is something that’s used in personality psychology. There’s a tremendous amount of psychological literature on it. Researchers went through thousands of words and figured out what words are related to certain personality types. Getting an appreciation for what’s behind all that was really interesting.”

The app immediately returns a graph showing the average prominence of core personality traits among people who use specific keywords.

“The app will get the most recent tweets that have your search query in it, and puts them through IBM Watson,” Agrawal explains. “You get back this huge JSON object with 55 different properties, and ten of those properties fall under the Big Five personality traits. You basically get a personification for the state as a whole.”

“Watson also provides the end user with a powerful translation feature,” Al Joubori adds. The group used this feature to add a translation feature to their app. Al Joubori speaks Arabic, and he was able to confirm Watson’s abilities in accurate, natural-sounding translation.

Making the front page of Hacker News was an exciting, educational experience for the team.

“One of the bigger takeaways was it’s important to see the wider scope of anything you’ve built,” notes Agrawal. “There are so many things that developers can use this fusion of Twitter and Watson for. It can have use-cases in marketing, for instance, or brand management.” He also mentioned the possibility of studying the correlation between how people talk about a company and the volatility of that company’s stock.

IBM has only recently released these features of Watson for use by developers, and it’s likely that trend will continue.

“They were literally adding features while we were building our project,” says Elauria.

In addition to its usefulness for the developer community, Personify provided proof of what this team could learn to do in just three months.

“I came to Hack Reactor having built nothing with JavaScript,” says Al Joubori. “Building the whole app, the whole library with the guys, my confidence level is up to the roof, which is amazing. It was a great experience.”

The library was built using Angular, Node, D3, Express, Gulp, jQuery, BootstrapKarma and Jasmine, as well as the Watson and Twitter APIs.

Want to grow your coding skills in the most effective program available? Apply to our program, online or onsite, today.

Read more:

Remote Beta Students Top Hacker News

Online Students Create WebRTC Library for Video Conferencing Apps

How Hack Reactor’s Online Immersive Coding Program is Different From Any Online Learning You’ve Experienced