Welcome to the Bulletin by Remix Robotics, where we share a summary of the week's need-to-know robotics and automation news.

In today's email -

• Why a robot’s reach exceeds its grasp (Big Idea)

• New machine learning hardware

• Lots of new investment news

• Hot takes on Amazon iRobot acquisition 

• A free training course in the UK

Snippets

MIT’s new machine learning hardware - Engineers working on hardware-based deep learning have developed a method of propelling protons through solids at unprecedented speeds, whilst utilising less energy. Why do they want to do this? Even on the most advanced computers, Deep learning algorithms are relatively slow as the code must constantly load and transfer data. A physical chip could do this much faster. In computer-based neural networks, algorithms mathematically weight connections between neurons. Now, MIT researchers have found a way to do this in a hardware analogue, by developing a programmable resistor which can still pass information millions of times faster than the human brain. This is a stepping stone to a physical Deep Learning chip, which would massively speed up machine learning tasks.

Trying to Top Tesla Tech - Jidu Auto, a joint venture by Baidu and Geely, has unveiled the concept for its first smart vehicle. Featuring LIDAR sensors, full voice control and no door handles, the concept vehicle is designed to be the most intelligent electric vehicle on the market. Baidu’s CEO -laid down the gauntlet and called Tesla out directly.

Robotic Bambi - In a scene familiar to those who have witnessed the first steps of a fawn, an AI-equipped robot dog at the University of California, Berkeley taught itself how to walk in an impressive 60 minutes. The algorithm — named Dreamer — is able to project the probability that a future action will achieve its goal, allowing for a quicker learning process. The researchers have already applied the technique to other applications, including pick and place (see this week's Big Idea).

(Re)-Shoring Up UK Manufacturing - David Nieper (a Derbyshire-based clothing manufacturer) —frustrated by disruptions to global supply chains — is emerging as an early proponent of the re-shoring. The company is putting its money where its mouth is and is investing in local communities, focusing on reskilling to allow British manufacturing to become competitive once again. Whilst many doubt the financial viability of fully UK-based manufacturing, this study highlights that the price of high-quality goods can be sufficient to absorb increased labour costs, with benefits to product quality and delivery.

Free “Introduction to Robotics” workshop for SMEs - Engineers from the Bristol Robotics Laboratory are holding a workshop in September for SMEs across the UK’s South West. It's a great way to develop new skills and we recommend calling into Bristol’s iconic Lakota afterwards — you won’t regret it.

Popping and Locking - Researchers at ETH Zurich have developed multi-stable building blocks that allow the construction of actively controllable shapes. These work by using very small pneumatic actuators to cause the structures to “snap-through” to another shape. This sounds much more complex than it is — if you’ve ever used a snap bracelet, you’re basically there!

The Big Idea

Why can’t Robots just get a grip?

The last few weeks we’ve been tip-toeing around one of the biggest bottlenecks in robotics - picking up and placing random objects. Last week, we discussed Moravec’s Paradox: how tasks that seem very easy for humans can be incredibly challenging for robotics. Nothing epitomises this more than universal grasping -

Universal grasping is the ability to pick up and manipulate a previously unknown object the first time it's seen, without any mistakes.

Grasping is one of the last steps required to bring robotics into parity with humans. Today, the majority of interest in grasping has been driven by E-commerce: an industry where almost every stage has been automated except picking and packing. This stage is currently very challenging due to the sheer variety of products. For example, Amazon has around 12 million unique product variants. But it’s not just e-commerce: Dyson is interested in universal picking for home robotics, and Google has a spinout company which is targeting pretty much every industry imaginable.

Universal picking is such an important topic that we’re going to spend the next few weeks reviewing its challenges and opportunities -

• This week we’re setting the scene and understanding the scale of the challenge.

• Next week we’ll explore the state of the art in robot picking + suggest where the future is trending.

How should universal grasping work?

We’re talking about robot systems, so no surprise universal gripping is tackled by the usual suspects - a robot, grippers, sensors and a control system. We’ll discuss the dominant technologies for each next week. If we abstract away from specific technological solutions, we can say there are 4 stages to grasping: planning the route, grasping the object, manipulating it through space and successfully placing it in the desired location. Although simple in principle, in practice an autonomous system must perform numerous complex tasks in each stage. We’ve broken down the main ones in this chart -

Why it often doesn't work

It may seem like we're overcomplicating things, but in industry (or any dynamic environment) major issues can emerge in all four stages. Generally, the problems originate from limitations in perception and control. Outside of a lab, systems struggle to capture and process the level of detail required in real-time. There is always a difference between the controller's view of reality and reality itself. It turns out that the grasping of objects is actually very sensitive to change, and so these small (in the best case scenario) differences compound into a large impact on success rates.

To match a human, a system needs to hit < 2 sec cycle times for the PnP of a never-been-seen-before (NBSB? - ok we'll stop) object. Currently, this isn't enough time to accurately -

• Identify an object's properties - e.g. centre of mass, friction, hardness, stiffness, number of moving components etc etc

• Model an object’s behaviour due to its properties - will it slide on the surface it's being held on if touched, will it deform when gripped, can it support its own weight etc etc

• Map the environment and keep track of it as the object moves in real time

As a result, in Stage 1 perception systems are still pretty poor at identifying new objects from backgrounds/other objects, automatically determining the best type of gripper to use, the best pick location for the gripper and how to move into position without collisions.

If Stage 1 goes well, Stage 2 is usually pretty smooth. Any mistakes though, and all bets are off. Issues can crop up if the gripper collides with other objects or accidentally unbalances the object it's trying to pick.

The first two stages are where researchers have focused the majority of their efforts, and — as we’ll see over the next few weeks — where there has been the greatest success to date. Unfortunately, the problems really start in the last two stages.

In Stage 3, systems can struggle to transport an object without collisions with the robot itself and other aspects of the environment. Any changes in the object’s positioning within the grips can throw the system off, and while solutions for avoiding the entanglement of objects exist, no general solution has been proposed to unhook complex object geometries.

Finally, in Stage 4, simply dropping objects, or even throwing them is straightforward. However, accurate placement is tricky on a flat surface, let alone into a container. The friction between a part and the grippers can change how an item falls, as can its centre of mass etc. Ensuring collisions are avoided and delicate objects are not crushed in real-time has not received sufficient attention to date, and so remains a large barrier to industrial deployment of intelligent pick and place for NBSB objects.

How humans solve the issue

To put the challenge into context, let's look at how the human body is shaped to deal with the challenge. Try to reframe all of these “features” as if you had to design an electromechanical system with the same capabilities.

Sensing

Human fingertips are probably the most sensitive skin areas in the animal world; they can feel the difference between a smooth surface and one with a pattern embedded just 13 nm deep. they’re able to differentiate between a wide range of textures, materials, temperatures, and pressures. They can sense pressure as light as a feather while being robust enough to handle loads of 20kg, high temperatures, abrasives etc. Our sense of touch is so responsive it can bypass our brain with the reaction to touch stimuli occurring in just 15 milliseconds.

Cameras are actually very similar to eyes (which makes sense - we copied them) but the resolution of the human eye is 576 megapixels in 3D - hard to beat. Even if standard machine vision cameras reach this level it's our brains that really set human vision apart. And speaking of brains…

Control

To understand our brains it's useful to contrast them with computers:

Speed - the brain can perform at most about a thousand basic operations per second… about 10 million times slower than then an average computer.

Precision - A computer can represent numbers with much higher precision. For example, a 32-bit number has a precision of 1 in 2^32 or 4.2 billion. In contrast, evidence suggests that most brains have a precision of 1 in 100 at best, which is millions of times worse than a computer. We also have a short-term working memory of around seven digits - Ok Computer, you win this one.

That said, a professional tennis player’s brain can follow the trajectory of a tennis ball at 160 miles per hour, move to the perfect spot on the court, position their arm and return the ball to the opponent, all within a few hundred milliseconds. And it can do this using ten times less power than a personal computer.

How does the brain achieve this while being so computationally limited? Computers perform tasks largely in a sequential flow. High precision is required at each step, as errors accumulate and amplify in successive steps. The brain also uses serial steps for information processing. In the tennis return example, information flows from the eye to the brain and then to the spinal cord to control muscles in the body. What’s key though is that the brain can undertake parallel processing at a scale that is hard to comprehend. First, each hemisphere can process independently. Secondly, each neuron collects inputs from — and sends output to — nearly 1000 other neurons. In contrast, a transistor has only three nodes for input and output. This parallel information processing capability means that humans are incredibly good at course correction: we can take information ‘on-the-fly’ and calculate how to adjust our movement strategy much, much faster than computational simulations.

It allows us to undertake all of the calculations and planning of incredibly complex tasks — like picking up a cup — without even realising.

The Manipulator

Our brains may be impressive, but we knew that already. The complexity of the human ‘end effector’ is what’s really stunning. On each hand, our wrist, palm, and fingers consist of 27 bones, 27 joints, 34 muscles, and over 100 ligaments and tendons. This achieves 27 degrees of freedom: 4 in each finger (3 for extension and flexion and one for abduction and adduction); the thumb is more complicated and has 5 DOF, leaving 6 DOF for the rotation and translation of the wrist. This flexibility gives us 33 highly optimised grip types - each for different situations and objects based on the mass, size and shape, rigidity, and force and precision required.

This sets the bar pretty high and may make the universal grasping problem feel insurmountable. Don’t let this put you off though — real progress is starting to be made. Next week, we’ll see how.

Interesting Investments

The big one - Amazon buys iRobot for $1.7bn - This shouldn't come as a surprise to any of you, it's the hottest piece of news this month. 3 takeaways -

• It’s all about data - iRobot is considered a leader in the indoor mapping of houses. Amazon already knows intimate details about what people eat, buy, watch and read. Soon, it may also know every inch of their homes - using floor plans, video recordings and object data, Amazon will be able to monitor when you’re in and out, map your home habits, collect data on your furniture and a whole lot more.

• It's not an amazing return for shareholders - 1.7B represents a 3% annualized return since the company went public in 2005, which doesn't even match inflation. That said, it's still a big deal for robotics companies and is Amazon’s fourth-most expensive purchase.

• It may not go ahead - Amazon is currently on the US government’s naughty list, and this purchase is drawing unwanted attention from the FTC.

Intelligent City raised $22M - The housing technology company applies automation and robotics to the design and manufacture of mass timber buildings. (Founded 2008, $30M raised to date)

Intuitive Surgical plans $100M investment - The US-based robotic surgical system manufacturer, will invest over 700 million yuan ($103.7 million) to build a manufacturing and innovation base in Shanghai to expand access to robotic-assisted surgery to more patients in China. (Founded 1995, $5.71B revenue in 2021)

Geek+ raises further $100M - The Beijing-based warehouse robotics firm has labelled this a “Series E1” raise, following its undisclosed Series D raise last year. (Founded 2015, $2B valuation)

Terabase Energy raised $44M - The solar technology company announced its Series B investment. This funding will be spent on designing a robot platform for the development, construction, and operation of their power plants. (Founded 2019, $52M raised)

Beyond Imagination enters agreement with SELF Labs - Beyond Imagination is to provide 1,000 humanoid robots to for use in off-the-grid agricultural “grow boxes”, equipped with solar panels, windmills, atmospheric water generators and 5G. We’re not sure what to think about this one. (Founded 2017, £13.2M raised)

Video

We’re starting a new segment called Amateur Hour - these DIY projects are getting out of hand.

This rocket project is more complex than a lot of industrialised products and demonstrated some serious dedication. If you think home projects are frivolous, pair it with this podcast interviewing the founder of Relativity Space, $4 billion dollar rocket company. He got started building space-worthy model rockets at university.

Image of the Week

Ok you may have noticed we won’t (can’t?) stop talking about AI-generated images… Well, we had a play with Mid Journey. It's not quite as good as Dalle-2 but it's free and there is no waiting list! (Yes, that's a Robot Remixing 🤖)