The JVM Option For Deep Learning When You Are Not Google Or Netflix

Realising the potential of large datasets through Deep Learning, for some, may be considered the remit of an exclusive club of organisations with deep, deep pockets and the infrastructure to assemble dedicated teams to venture into areas of research which have not yet been explored by humans.

Not so, according to Adam Gibson, Founder of SkyMind, the company which supports the open-source projects which are advancing scientific computing on the JVM, DeepLearning4J and ND4J. Gibson is living proof that Deep Learning is tangible if the inclination is there, “I dropped out of school. I’ve always been self-taught so it mainly comes down to the fact that you just need motivation and grit. It was just a combination of me reading papers and implementing concepts and repeatedly making mistakes until I learned what I needed to learn. I have an implementers perspective on a lot of things, I like to test things just like any engineer and figure out ‘how does this work, what does it do and decomposing things.'”

Nowadays, Gibson undertakes a lot of consulting work for Fortune 500 companies to make sense of the pools of data they are sitting on, “We typically work on solving problems with enterprise outfits that run Hadoop where developers may not have access outside the firewall and they need something that is a first class citizen on their cluster and we work more with Fortune 500 companies rather than startups in this case. There are plenty of Javascript accessible things out there, we’re bridging into Python and some other libraries “.

In terms of the types of problems which they approach, Gibson goes on “Humans are really good at pattern recognition, sight, reading things basically ingesting patterns. Specifically unstructured data, video images, text.. things that we see maybe a Data Scientist has a hard time taking apart the features for so I mainly help these companies make sense of unstructured data and usually they have a lot of it in the Hadoop file system and they don’t know what to do with it. We kind of help them bridge that gap.”

Gibson also explains how he made the choice to run on the JVM and what it has going for it, “If you think about it what our production data stacks written in? They are written on Hadoop, they are written on Spark, they are written in Scala, modern data pipelines which are on the JVM. Why not to be a first class citizen right where the data lives in the same process, there is no reason we can’t do that.”

“A lot of problems though with machine learning up until now with the JVM have just been about the fact that there hasn’t been a library that gives data scientists familiar syntax so I drew a lot of comparisons between me teaching Python to Data Scientists as well as training other experts on Java, I drew comparisons in figuring out what the JVM was missing and one of those things was another matrix library..   why not solve it once and for all and give data scientists a library that is future proof with familiar syntax.”

I terms of the approach with learning how to drive Deeplearning4J, the advice is to dive right into -core, “We have a few example projects out there but I would still highly recommend playing with core and actually understanding the architecture so there are actually two major modules, -core which is the core algorithms, you can run them, play with them, I usually just run the tests. What I want to do is put out more curated examples for people to play with right now. I just have a lot of examples derived from unit tests, not highly tuned but it shows you example usage. We have a very well unit tested code base and I recommend just diving into cause understanding whether layers are the core API and then working from there scale out is just parameter averaging so it’s just averages or what ever you are going to do in -core.”

Gibson was feature speaker at February’s San Francisco Spark Meetup where he gave a talk on DeepLearning4J on Spark comprising an introduction to nd4j, the core algorithms in deeplearning4j and distributed Deep Learning, using Jeff Dean Style Parameter averaging.

Comments

comments

About Gary Donovan

Machine Learning and Data Science blogger, hacker, consultant living in Melbourne, Australia. Passionate about the people and communities that drive forward the evolution of technology.
Show Buttons
Share On Facebook
Share On Twitter
Share On Linkedin
Share On Pinterest
Share On Stumbleupon
Contact us
Hide Buttons