The Complete Developer’s Guide to Intel AI Resources & Tools (Portfolio)
Often during most of Intel’s Student Developer Program workshops and trainings that I conduct, aspiring developers are always puzzled at how they can get started on Artificial Intelligence in a fine-drawn way. Any developer getting started on the AI field may be lost on where to begin, given the industry is continually getting new entrants, resulting to a large array of products competing for dominance in the AI and deep learning space.
AI has surged in popularity, with its resurgence far exceeding its past glimpses of popularity. The exponential increase in AI can be majorly attributed to availability of large data sources, growth in distributed computing systems, and modern algorithms development based on neural networks.
The delivery of AI solutions is being partially fueled by a breakthrough in compute performance. With today’s compute capability and architectural innovation, developers can now meet the intense demands of artificial intelligence. Intel has continually been a big force in pushing and accelerating the advancement of AI and its subsets, i.e. Machine Learning and Deep Learning. Intel is working diligently to provide the compute building blocks that are going to drive Artificial Intelligence. Intel AI focuses on accelerating developer’s path of turning data into intelligence.
As Andrew Ng (Co-founder, Coursera; Adjunct Professor, Stanford University) famously put it in AI is the next electricity: “Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years.”
AI has the power to change society for the better, and thus Intel has committed to three key areas: Developing a broad product portfolio benefiting from the advancements of Moore’s Law, investing in training and resources to help make AI technologies accessible for everyone, and fostering beneficial uses of AI technology through partnerships and investments.
INTEL END TO END AI PORTFOLIO
Intel offers the most comprehensive and diverse portfolio of products for machine and deep learning from the data center to the edge. By providing compute resources for machine learning and deep learning training and inference compute needs, Intel is enabling users to start exploring AI innovation without making their own investments in compute resources up front. In addition to cloud compute resources, frameworks, tools and support are provided.
Down the road, the Intel Nervana hardware will deliver unparalleled deep learning performance and efficiency that dwarf general-purpose GPUs.
Developers can leverage on Intel’s AI product portfolio to accelerate their AI solutions:
1. Intel Xeon Processors
Intel processors fuel a vast majority of servers in data centers (>97%). In a market for so-called “accelerator” chips, Intel continually bursts myths on training and inference on CPUs lagging behind GPUs. Intel has a portfolio of AI technology for the data center, all brought under their Intel Nervana AI Portfolio for synergy. It includes software tools and framework optimizations to deliver exceptional performance for data-intensive workloads to complement the Intel Xeon Scalable processor (which meets the Machine Learning and Deep Learning needs of most enterprises), Intel Xeon Phi processor, AI accelerators like Intel FPGAs, and AI-specialized Intel Nervana™ technology (codenamed Lake Crest).
i) Intel Xeon Scalable Processors : Designed with advanced AI and HPC capabilities, including significant increases in memory and I/O bandwidth, data scientists are able to unlock insights faster from data-intensive workloads. It has the capability to accelerate product innovation with its advancements across compute, storage, memory and I/O, which will significantly advance AI. It delivers up to 2.2X higher deep learning training and inference performance over prior generations. Recently, Intel announced the availability of the Intel Xeon processor E3–1200 v6 product family, which includes eight new versions that provide key improvements in processor speed, enhanced memory capabilities, hardware-enhanced security and reliability features with support for the latest Intel graphics.
ii) Intel Xeon Phi Processors : Intel’s upcoming generation of Intel Xeon Phi processors (codenamed Knights Mill) will deliver significantly faster throughput for training complex neural networks for workloads that require dense compute and very high levels of parallelism.
iii) Intel FPGAs : FPGAs offer the most promise for inference, as they can be upgraded while in the field and could provide low latencies if located at the edge alongside a CPU, such as in an autonomous vehicle. Intel Arria and Stratix FPGAs function as programmable hardware accelerators for specialized deep learning inference processing when combined with Intel Xeon processors.
Through the Intel Student Developer Program, developers can gain direct access to their own instance on the Intel® Xeon Phi™ powered AI cluster, to power the development and training of deep learning models.
2. Intel Libraries
Intel has developed highly optimized math libraries that make the most efficient use of the several families of Intel processors. Those optimized math libraries are the foundation for higher-level tools and frameworks that are used to solve ML and DL problems across different domains.
Intel Math Kernel Library (Intel MKL) : includes a set of highly-optimized performance primitives for DL applications, as well as functions that have been highly optimized for single-core vectorization and cache memory utilization
Intel Math Kernel Library for Deep Neural Networks (Intel MKL-DNN) : This library provides optimized deep neural network primitives for rapid integration into DL frameworks. It includes highly vectorized and threaded building blocks for implementing convolutional neural networks with C and C++ interfaces. Useful links to get started on MKL-DNN:
Intel Data Analytics Acceleration Library (Intel DAAL) : helps speed big data analytics by providing highly optimized algorithmic building blocks for all data analysis stages (Pre-processing, Transformation, Analysis, Modeling, Validation, and Decision Making) for offline, streaming and distributed analytics usages. It’s designed for use with popular data platforms including Hadoop, Spark, R, and MATLAB for highly efficient data access.
3. Intel Optimized Frameworks
Software innovation is key to taking advantage of features in underlying hardware. It is also key to accelerating the development process for analytics and HPC applications.
Beyond silicon, Intel is committed to optimizing the most popular analytics, machine & deep learning frameworks including Tensorflow, Caffe, Torch, Theano, and Spark-based MLlib and BigDL, so that they run best on Intel Architecture (IA).
A convenient way to work with Deep Learning is to use the Intel’s optimized ML and DL frameworks. Using Intel optimized tools and frameworks to train and deploy deep networks guarantees that these tools will use Intel architecture in the most efficient way. Examples of how some of these frameworks have been optimized to take advantage of Intel architecture, as well as charts showing speed-up of these optimized frameworks compared to non-optimized ones, can be found in https://software.intel.com/en-us/ai/deep-learning
The frameworks use the functionality of the libraries described above. They allow developers to perform training and inference in a highly efficient manner using Intel processors.
4. Intel Deep Learning Training Tool
Since mid-2017, Intel ambassadors have been holding workshops in various universities and corporates around Kenya, as part of the larger worldwide ecosystem of the Intel Student Developer Program. Central to these workshops has been the training of the Deep Learning Training Tool from Intel, which to say the least is heaven-sent for any developer or data scientist. And there are numerous reasons to it.
Most data scientists and developers don’t yet use deep learning, due to the relative complexity of customizing deep learning models for their own problems, the challenges in installing and using the required frameworks, and the low performance of open source deep learning frameworks on standard CPUs. This has been ground to a lot of constraints in democratizing AI.
The solution to the complexity is Intel’s Deep Learning Training Tool. The Training Tool is a web application running on a Linux server and provides a user-friendly, intuitive interface for building and training deep learning models. The best part of the tool is how easy it is to train and deploy deep learning models for image and text analysis problems. Testing it out with the famous MNIST dataset, training was achieved within less than 30 minutes, a task that would initially take more than double the time. Intel’s Deep Learning SDK includes tools for easy data preprocessing and model compression, providing an end-to-end solution. In addition, it supports scale-out on multiple computers for training, as well as using compression methods and optimization for deployment of the models on various Intel hardware platforms, including those with limited memory footprints and performance constraints.
It provides:
- The ability to visualize real-time data — without programming — during the creation of deep neural networks
- Simplified installation of deep learning frameworks that are optimized for Intel architecture
- Integrated Jupyter* Notebooks for advanced model creation
- The visual means of setting up, tuning, and running deep learning algorithms with the model wizard
It is still currently in beta version, although it is confidently stable gauging from past training and inference actions.
For a comprehensive installation guide, check out Marvin Ngesa’s blog.
5. Intel Nervana DevCloud
Intel Nervana DevCloud is a cloud-hosted hardware and software platform for developers, data scientists, researchers, academics and startups to learn, sandbox and accelerate development of AI solutions with free compute cloud access powered by Intel Xeon Scalable processors. By providing compute resources for machine learning and deep learning training and inference compute needs, Intel is enabling users to start exploring AI innovation without making their own investments in compute resources up front.
It will be available to 200,000 developers, including startups from Intel’s Nervana AI Academy, through a simple registration process.
6. Movidius Neural Compute Stick
As Intel pushes forward with machine learning development by way of edge devices, it acquired Movidius, a chipmaker company leading in the VPU (Vision Processing Unit) platform for on-device vision processing. After their acquisition, they have announced an array of products that give power to inference on the edge:
i) Movidius Neural Compute Stick (NCS): The Movidius (NCS) is a tiny fanless deep learning device that you can use to learn AI programming at the edge. As Intel states, the Movidius NCS is “the world’s first self-contained AI accelerator in a USB format,” and is designed to allow host devices to process deep neural networks natively — or in other words, at the edge. In turn, this provides developers and researchers with a low power and low cost method to develop and optimize various offline AI applications. Under the hood, the Movidius NCS works by translating a standard, trained Caffe-based convolutional neural network (CNN) into an embedded neural network that then runs on the VPU.
ii) Myriad X: Myriad X is the first VPU to feature the Neural Compute Engine — a dedicated hardware accelerator for deep neural network inferences. The Neural Compute Engine in conjunction with the 16 powerful SHAVE cores and an ultra-high throughput intelligent memory fabric makes Myriad X the industry leader for on-device deep neural networks and computer vision applications.
7. Intel Saffron
Saffron Technology develops cognitive computing systems that use incremental learning to understand and unify by entity. Saffron’s unique technology ingests data from disparate sources and automatically connects the dots to help businesses of all kinds improve decision-making. It captures the connections between data points at the entity level and stores these connections in an associative memory.
The interesting bit is that when machine learning logics and Saffron’s reasoning systems are fused, intelligent local analytics becomes possible in the Internet of Things.
8. Trusted Analytics Platform
Trusted Analytics Platform (TAP) is a set of open source tools designed to make big data projects easier to implement. TAP includes a series of algorithms, tools and engines to analyze and collaborate with data scientists. The software is open source and includes integration with Intel architecture and hardware. TAP includes:
· A data layer that includes Apache Hadoop and Spark optimized for Intel hardware.
· An open analytics layer that can support predictive application programming interfaces.
· Support for cloud native apps.
Intel is highly committed to democratizing AI by providing these tools to developers interested in diving into development with AI. AI is still in its infancy, and as this space evolves, Intel will continue to advance disruptive approaches in compute that support the complex workloads of today and tomorrow. This article will be regularly updated as Intel adds AI products onto its portfolio.
Chris Barsolai is an Intel Student Ambassador with the ISA Program. Find him on LinkedIn or Twitter for more articles and updates on the AI scene.
