In my last post I wrote about ways to come up with ideas for side projects and that idea generation is really just problem solving in disguise. I was finally able to cook something up and, interestingly enough, it was the result of a number of random things occurring at the same time and it didn’t end up being what I thought it would be. Creativity, in the traditional sense, had nothing to do with it. An unintended consequence of the exercise was that I started to change my mindset about learning and applying technology so there’s a moral to the story…
For the nuts and bolts about how things came together:
I was trolling through Reddit r/datasets and I came across the data from the TSA that has information about domestic commercial flights. When I say information I mean everything about a flight: the carrier, scheduled departure, actual departure, scheduled arrival, actual arrival and so on. (This is not the source I used but it is the same data through 2008). I thought I would apply general data mining/data science techniques to get information about flight delays and do/learn about visualization techniques. This problem has been tackled already so if I get stuck there are resources to fall back on.
OK, sounds pretty vague so far.
The other day I got a call from my boss asking me about doing a chatbot type thing. I found the idea intriguing and thought maybe, for my side project, I could use IBM Watson Conversation and give insights about the airline flight data in a conversational format. OK, things are taking shape.
Soon after, I got an email from a meetup group I joined a couple of years ago but never went to, GDG Hudson Valley. It is focused on Google and Android and I’m not an Android/Google developer. The group was starting a machine learning camp and python is the order of the day so I went. Before we got to the part about machine learning there were talks about Google I/O and Google Assistant. It turned out that the Google Assistant topic was very interesting since it is all about communicating with a computer using speech. Hmmm.
To tie things back to my last post about idea generation being an exercise in problem solving:
In the end, by thinking about a use case and keeping my eyes and ears open, I’ve come up with a pretty good direction forward. I’m going to focus on the machine learning study group and do some work with Watson Conversation and Google Assistant.
To get back to my last post, I wrote that:
- Use cases/problem sets don’t have to be original. Airline flight delays is not original so check.
- I wrote that there are plenty of free data sets out there. TSA flight data, check.
- Networking and meetups are good sources of inspiration. Machine learning study group, check. This actually works out better since it is ongoing and there are other people involved.
The moral of the story, or unintended consequences:
The process took a bit longer than I thought it would but, over the last couple of weeks, project idea generation has always been in the back of my mind and, because of that, there was a kind of filter through which I saw things. I wasn’t thinking in terms of specific technologies or platforms or tools but rather in terms of how I could apply “technology” in a generic sense. (“I’m deciding what to have for lunch. How could I solve this problem better? LOL.) An email for an online course or meetup, a conversation, all of it provided fodder for ideas. I didn’t care what the idea was and I wasn’t thinking about specific technical details since, like I wrote last time, take care of the use case and the technology will take care of itself.
This might seem like a long way to go for a pretty basic idea but, as they say, it’s not the destination but the journey. By just coming up with an idea in the manner in which I did I’m better at what I do than I was a couple of weeks ago. Also, when I carry this simple idea to its conclusion I’ll be a better software developer than I am now. I’m going to keep going down this way of thinking and, over time, I’ll get better at it and I’ll probably even come up with some good ideas.