PRO TALK (AI): Tensorflow Inference on WebAssembly — From 10 Minutes to 10 Milliseconds

AI DevWorld -- PRO Stage 4

Michael Yuan
Second State, CEO

Second State provides comprehensive blockchain solutions for businesses. Based on a full stack of Open Source software, it creates turn-key blockchain solutions and subscription-based services optimized for specific industries. Second State has recently closed a $3 million A round venture financing led by SIG.

Dr. Michael Yuan is the CEO of the Second State Inc, and a co-founder of the CyberMiles Foundation. He is the author of textbook Building Blockchain Applications published by Addison-Wesley. Dr. Yuan has extensive experience developing and commercializing Open Source software. His past experience includes product management and developer programs in major open source companies such as JBoss and RedHat. Dr. Yuan received a PhD in Astrophysics from the University of Texas at Austin.

This presentation will cover options to run Tensorflow model inference on WebAssembly. We will start from the unique challenges of deploying AI inference models in production, and how Rust + WebAssembly could help. Using a Mobilenet image classification task as an example, we will discuss the pros and cons of the plain JS approach, Tensorflow.js, pure Rust crates for Tensorflow compiled to Wasm, and WASI-like Tensorflow Wasm extensions that run on specialized inference chips. We will go through the journey of 60,000x performance gain over different WebAssembly approaches. We will also discuss what’s the future for WebAssembly-based AI on the edge cloud.