MemryX is a new AI company we really need
Artificial intelligence (AI) is all around us, from the cars we drive to the smartphones we use, powering our most beloved or most despised apps. Therefore, AI is one of the main growth drivers for the semiconductor industry. It is everywhere and is revolutionizing the way humans interact with the modern world, from the data center to the device edge and everything in between. Running AI at the edge has become commonplace because these interactions need to happen quickly with low latency while minimizing data transport expenses.
MemoryX, a new startup specializing in AI processing for peripheral devices, is creating new technologies to address this emerging market. But do we really need another AI chip company? I think we’re still in the second or third round of this game and MemryX has the potential to disrupt the device edge semiconductor market with a combination of its unique AI architecture, focused on memory and its easy-to-use and scalable software/hardware solution. The company’s management is both experienced and hands-on, knowing what it takes to develop high-volume semiconductors. If you’re not first to market, you have to be very different and that’s where I see MemryX fitting in.
Let’s take a closer look at MemryX, its direction, and how it might be different from the pack.
MemryX was co-founded in 2019 by Dr. Wei Lu, MemryX CTO, to develop a leading AI accelerator for Edge devices. Dr. Lu has been a professor of electrical engineering and computer science at the University of Michigan since 2005. His expertise in memory systems and neuromorphic computing is internationally recognized. Dr. Lu and his team spent nearly three years developing and proving MemryX’s unique approach.
In December 2021, MemryX hired Keith Kressin as President and CEO to commercialize MemryX’s technology. Keith has over 25 years of experience in semiconductor leadership and was previously senior vice president and general manager at Qualcomm. He helped guide technology and products at Qualcomm through key growth phases of 3G/4G/5G smartphones, AR/VR and AI initiatives. He also spent nearly a decade at Intel. I know Keith well and think he’s the right guy to scale a business. Keith’s reputation has enabled him to hire a strong leadership team in engineering and sales/marketing, bringing decades of leadership experience at AMD, Intel, Micron and Qualcomm. Experience matters.
The company has been frugal, having raised a very modest $11 million in seed and Series A, while successfully registering multiple generations of chips to prove the technology works. It just recently announced a customer sampling of its pre-production MX3 AI Accelerator hardware and software. I’m sure Series B isn’t far behind given the financial levels required to be in the increasingly strategic semiconductor market.
The plant target and flag
Unlike most AI startups that claim to fix everything for everyone, MemryX is focused on providing an Edge AI inference solution (not training) for vision and sensing solutions. Its solution is an AI accelerator that complements, rather than replaces, a central processor for new or legacy systems. So he’s trying to do one thing really well. Overall, MemryX puts the flag on providing a high-efficiency, low-power solution that benefits the end user. But often, more importantly, they provide an Edge AI solution that is the easiest to implement, which benefits the designer.
The problem MemryX is trying to solve is a multi-variable challenge.
The growing diversity of smart devices makes it difficult to use traditional computing architecture to run AI models at the edge. Moreover, AI systems are data-centric, while conventional CPUs are designed to execute instructions. In traditional computing, AI models are stored in DRAM, resulting in bottlenecks, higher latencies, and higher power. So many systems can run AI, but not very efficiently.
AI accelerators for peripheral devices are expensive to develop due to the difficulty and inconsistency of software implementation. It typically takes considerable engineering effort to create and update AI algorithms on a growing range of hardware solutions, including ARM, x86, and now RISC-V ISA. And just as investments are made, continuous changes to algorithms and operating systems continually increase the need for additional investments.
As AI makes its way into almost every chip and device, it’s becoming apparent that dedicated AI accelerators have limited traction in the market. A new approach to cutting-edge AI that solves these problems could add tremendous value.
A solution with ‘One Click Optimization’
By evolving to a single dataflow architecture with in-memory computing, much like how AI networks are naturally organized, MemryX should overcome software implementation bottlenecks and complexity and meet the needs of Edge AI on a larger scale.
MemryX’s Neural Processing Unit (NPU) has lower latency than other AI accelerators due to its inherent ability to store AI models on-chip rather than in DRAM. The proprietary data flow architecture is also designed to minimize any movement of data within the chip, as moving data requires more power than AI calculations. This allows their solution to maximize throughput while minimizing power consumption.
The architecture enables “one-click performance optimization” delivering 50-70% chip utilization without the need for manual software tuning. I admit I was a bit skeptical when MemryX informed me about this. It was almost too good to be true considering that manually tuning AI models for weeks or months is common practice. I became much more convinced when the company shared with me a growing list of over 100 templates in 13 categories that work effectively on their solution all using the same software. It also supports models in all popular AI frameworks. Meanwhile, MemryX has only 30 employees. Impressed yet?
The company’s latest chip, the MX3, is also scalable, so one can plug in anywhere from 1 to 16 small chips, scaling the production version of the chip from 5 TFLOPs to 80 TFLOPs. The performance per watt is even more impressive given that each chip consumes an average of just 1 watt of power. This means that MemryX, without software tuning, is more than 5 times more efficient than an NVIDIA AGX Xavier system that uses NVIDIA’s proprietary software. I haven’t personally performed the tests, but the testing methodology passes my smell test.
Move with momentum
MemryX is just over three years old and has made great strides for me, completing its seed funding round in 2019 and just seven months later proving that its proprietary architecture works in silicon. Last year, MemryX produced its pre-production chip, which it announced this week was being sampled to customers. The company expects volume production to begin in 2023.
Given that MemryX is producing such a focused product with the potential to scale in a way that hasn’t been seen much in the cutting-edge low-powered AI market, I think if all goes as planned, this could have a huge impact on on-board intelligence. MemryX claims to work with alpha customers in several target markets, including automotive, edge servers, metaverse, and security cameras. I believe this gives MemryX a diverse portfolio of markets and multiple opportunities for significant growth.
While some might say the AI accelerator space is crowded, it strikes me that MemryX is different. It uniquely solves well-known problems in cutting-edge AI. Its focus on unique technology to meet real market needs is half the battle. MemryX seems off to a good start.
I look forward to further disclosures and will share as appropriate.
Moor Insights & Strategy, like all research and technology industry analytics companies, provides or has provided paid services to technology companies. These services include research, analysis, consulting, consulting, benchmarking, acquisition matching and conference sponsorship. Company has had or currently has paid business relationships with 8×8, Accenture, A10 Networks, Advanced Micro Devices, Amazon, Amazon Web Services, Ambient Scientific, Anuta Networks, Applied Brain Research, Applied Micro, Apstra, Arm, Aruba Networks (now HPE), Atom Computing, AT&T, Aura, Automation Anywhere, AWS, A-10 Strategies, Bitfusion, Blaize, Box, Broadcom, , C3.AI, Calix, Campfire, Cisco Systems, Clear Software, Cloudera, Clumio, Cognitive Systems, CompuCom, Cradlepoint, CyberArk, Dell, Dell EMC, Dell Technologies, Diablo Technologies, Dialogue Group, Digital Optics, Dreamium Labs, D-Wave, Echelon, Ericsson, Extreme Networks, Five9, Flex, Foundries.io, Foxconn, Frame (now VMware), Fujitsu, Gen Z Consortium, Glue Networks, GlobalFoundries, Revolve (now Google), Google Cloud, Graphcore, Groq, Hiregenics, Hotwire Global, HP Inc., Hewlett Packard Enterprise, Honeywell, Huawei Technologies, IBM, Infinidat , Infosys, Inseego, IonQ, IonVR, Inseego, I nfosys, Infiot, Intel, Interdig ital, Jabil Circuit, Keysight, Konica Minolta, Lattice Semiconductor, Lenovo, Linux Foundation, Lightbits Labs, LogicMonitor, Luminar, MapBox, Marvell Technology, Mavenir, Marseille Inc, Mayfair Equity, Meraki (Cisco), Merck KGaA, Mesophere, Micron Technology, Microsoft, MiTEL, Mojo Networks, MongoDB, National Instruments, Neat, NetApp, Nightwatch, NOKIA (Alcatel-Lucent), Nortek, Novumind, NVIDIA, Nutanix, Nuvia (now Qualcomm), onsemi, UNOG , OpenStack Foundation, Oracle, Palo Alto Networks, Panasas, Peraso, Pexip, Pixelworks, Plume Design, PlusAI, Poly (formerly Plantronics), Portworx, Pure Storage, Qualcomm, Quantinuum, Rackspace, Rambus, Rayvolt E-Bikes, Red Hat, Renesas, Residio, Samsung Electronics, Samsung Semi, SAP, SAS, Scale Computing, Schneider Electric, SiFive, Silver Peak (now Aruba-HPE), SkyWorks, SONY Optical Storage, Splunk, Springpath (now Cisco), Spirent, Splunk, Sprint (now T-Mobile), Stratus Technologies , Symantec, Synaptics, Syniverse, Synopsys, Tanium, Telesign, TE C Connectivity, TensTorrent, Tobii Technology, Teradata,T-Mobile, Treasure Data, Twitter, Unity Technologies, UiPath, Verizon Communications, VAST Data, Ventana Micro Systems, Vidyo, VMware, Wave Computing, Wellsmith, Xilinx, Zayo, Zebra, Zededa, Zendesk, Zoho, Zoom and Zscaler.
Patrick Moorhead, Founder, CEO and Chief Analyst of Moor Insights & Strategy, is an investor in dMY Technology Group Inc. VI, Dreamium Labs, Groq, Luminar Technologies, MemryX and Movandi.