Realising the potential of large datasets through Deep Learning, for some, may be considered the remit of an exclusive club of organisations with deep, deep pockets and the infrastructure to assemble dedicated teams to venture into areas of research which have not yet been explored by humans.
Not so, according to Adam Gibson, Founder of SkyMind, the company which supports the open-source projects which are advancing scientific computing on the JVM, DeepLearning4J and ND4J. Gibson is living proof that Deep Learning is tangible if the inclination is there, “I dropped out of school. I’ve always been self-taught so it mainly comes down to the fact that you just need motivation and grit. It was just a combination of me reading papers and implementing concepts and repeatedly making mistakes until I learned what I needed to learn. I have an implementers perspective on a lot of things, I like to test things just like any engineer and figure out ‘how does this work, what does it do and decomposing things.'”
In terms of the types of problems which they approach, Gibson goes on “Humans are really good at pattern recognition, sight, reading things basically ingesting patterns. Specifically unstructured data, video images, text.. things that we see maybe a Data Scientist has a hard time taking apart the features for so I mainly help these companies make sense of unstructured data and usually they have a lot of it in the Hadoop file system and they don’t know what to do with it. We kind of help them bridge that gap.”
Gibson also explains how he made the choice to run on the JVM and what it has going for it, “If you think about it what our production data stacks written in? They are written on Hadoop, they are written on Spark, they are written in Scala, modern data pipelines which are on the JVM. Why not to be a first class citizen right where the data lives in the same process, there is no reason we can’t do that.”
“A lot of problems though with machine learning up until now with the JVM have just been about the fact that there hasn’t been a library that gives data scientists familiar syntax so I drew a lot of comparisons between me teaching Python to Data Scientists as well as training other experts on Java, I drew comparisons in figuring out what the JVM was missing and one of those things was another matrix library.. why not solve it once and for all and give data scientists a library that is future proof with familiar syntax.”
I terms of the approach with learning how to drive Deeplearning4J, the advice is to dive right into -core, “We have a few example projects out there but I would still highly recommend playing with core and actually understanding the architecture so there are actually two major modules, -core which is the core algorithms, you can run them, play with them, I usually just run the tests. What I want to do is put out more curated examples for people to play with right now. I just have a lot of examples derived from unit tests, not highly tuned but it shows you example usage. We have a very well unit tested code base and I recommend just diving into cause understanding whether layers are the core API and then working from there scale out is just parameter averaging so it’s just averages or what ever you are going to do in -core.”
Gibson was feature speaker at February’s San Francisco Spark Meetup where he gave a talk on DeepLearning4J on Spark comprising an introduction to nd4j, the core algorithms in deeplearning4j and distributed Deep Learning, using Jeff Dean Style Parameter averaging.