This week launched a new series, "Line By Line". By viewer request, I walked through the excruciating details of TensorFlow's fully_connected_feed.py script. It has a lot of little components that can trip you up if you're not used to seeing them.

The upside of this kind of video is that it's hard to totally screw up since it's vetted existing code. On the flip side, I can only do this with something I know really well. In this case, I've done the MNIST example in my sleep, so it's not a problem. If you've got other tutorials or topics you'd like exposed, let me know!

It's taken several weeks, but I finally got a simple RNN model working in Keras. The key was understanding that Keras wanted to work on one sequence at a time, not batch them all up. So I had to change up my training to iterate through all the sentence, doing one epoch at a time. Maybe there's a better way, but this finally worked and gave a model that did vaguely the correct thing.

After producing a working model for this word embedding problem, I wanted to have something simpler just to get a better grip on RNNs. So, I generated some fake sinusoidal temperature data and quickly grabbed some rainfall information off the web. I was able to use this to build a model to predict the season based on daily values of these (and history via the RNN). Doing this hacky thing doesn't work too well, but keep your eye out to if I revisit this type of data again.

Since the last video, Keras officially released version 1.0, woohoo! In this video, I poked around at some of the new features and changed syntax. Most of it is pretty much the same. The big change was that "Graph" objects are now "Functional" objects. I think I preferred the old name, but I never got around to using it, so I don't get a vote.

I also tried to get a simple RNN running again, but the data just isn't lining up for this. I think I need to find a minimal example, with the data spelled out, to see what my problem is. I need a "DanDoesData" for RNNs :p

I think I've goofed off with Keras enough for now, and it's time to move on to another topic. Several viewers have sent in suggestions, so I'll pick one of those from a hat.

For April Fool's day, I wanted to do something different, so I planned to pan the very tool that served me through undergrad: Excel97. As luck would have it, some other folks had the same idea and scooped me! So, I decided to go through my usual paces of learning a tool, but this time with Deep Excel (and still the MNIST model, of course)

Mostly, I malign spreadsheets because they're the wrong tool for the data science and machine learning job. They are great for some tasks, like tabularizing and doing simple computations on data. And Excel gives you the ability to easily see your variables as you manipulate them. But I still remember how much I fought Excel trying to implement something like a "for loop". This was before I knew Python or even MatLab; if only I'd known there was a better way.

I missed posting about last week's successful video defining custom Keras activations! This wasn't nearly as painful as figuring out how to access the weights in an skflow model, but it's about as much work after the fact. Like everything in life, no tool is perfect.

And just tonight I started looking at Recurrent Neural Networks. These are cool ways to introduce the element of time in your dataset. Essentially, you're adding extra features from the previous data point into your current time step's neural net. It gets more complicated than that with things like LSTM (use those features to decide when to "forget" old information and the like), but we're sticking to the basics.

Unfortunately, I didn't see a tutorial on using RNNs with Keras, so we're winging it off of a Theano tutorial. But it's a little light on the explanations and details. Hopefully my video starts to clear the waters.

In the end, I couldn't get the model working by the 1 hour mark. That's ok, this just gives us another topic to cover next week: Keras RNN 2, Electric Boogaloo.