Neon Genesis | The Juice

Zumo Labs presents The Juice, a weekly newsletter focused on computer vision problems (and sometimes just regular problems). Get it while it’s fresh.

Week of June 28–July 2, 2021

____

When it comes to generating 3D computer graphics, there’s no shortage of software options available. How you decide which software to use is generally priority calculus — creating meshes for an industrial use case? You may want CAD-specific software like AutoCAD. Modeling for animation? Perhaps Maya. Want to script your assets? Game engines such as Unity or Unreal work well for that.

But what if you want to do everything? And what if you’d prefer to do everything under an open source license (aka free)? Well, there’s an app for that. Today Hugo is sharing his thoughts on Blender, the 3D software ecosystem’s jack-of-all-trades and synthetic training data’s greatest ally.

____

#CardCatalog

The Library of Congress, established in 1800 to document American history, has 170 million physical items and a whopping 410 million file-digital collection. Its current search technology utilizes text and metadata indexing that omits millions of items from search results entirely. So the library is tackling the problem with a two-pronged approach, using computer vision to examine and identify thousands of images and a neural network to understand better the intent behind a user’s search and return the best and most extensive array of results.

Library of Congress Looks to AI to Help Users Sift Through Its Collection, via WSJ.

#ResponsibleAI

While it remains unclear how exactly regulators plan to police biased AI in the US and abroad, one certainty is that, sooner or later, many companies will need to audit and improve their existing systems and processes. That provides a big market opportunity for startups like Parity, a new compliance and governance platform. This week Parity announced that ethical tech advocate Liz O’Sullivan — known for her work on the Campaign to Stop Killer Robots — is taking over as CEO.

Using A.I. to Find Bias in A.I., via The New York Times.

#BigSleep

You’re probably familiar with the infinite monkey theorem, which says that if you let a monkey mash a keyboard long enough (for eternity), you’d almost definitely wind up with the complete works of Shakespeare. The theorem never considers the possibility that you’d wind up with something even better.** Anyway, the GPU equivalent of that is what we think about when we peruse the mind-boggling AI-generated artwork made with BigSleep from Ryan Murdock.

The Persistence of Memory by Lisa Frank, via Twitter.

#BodyLanguage

Ever had that awkward “are they going for a handshake or a hug” panic when you run into an acquaintance on the street? This robot from Columbia University might have less social anxiety than you. The AI, trained on thousands of hours of videos from sitcoms to sports games, can accurately predict human behavior in videos up to several minutes in advance. If it can’t make a guess, it ventures a broader prediction (“the people will greet”). Didac Suris, a Ph.D. engineering student and the paper’s co-author, knows “not everything in the future is predictable,” but “when a person cannot foresee exactly what will happen, they play it safe and predict at a higher level of abstraction. Our algorithm is the first to learn this capability to reason abstractly about future events.”

AI Learns to Predict Human Behavior from Videos, via Columbia University.

#Clearview

The U.S. Government Accountability Office launched a probe into federal use of Clearview AI and published their findings on Tuesday. The nearly 100-page report was based on internal Clearview data initially reviewed by Buzzfeed News, which believes that five agencies the GAO says didn’t use the software, did. Those agencies include the US Capitol Police, the US Probation Office, the Pentagon Force Protection Agency, Transportation Security Administration, and the Internal Revenue Service. The discrepancies remind us that when it comes to AI, sometimes your checks and balances need checks and balances.

A Government Watchdog May Have Missed Clearview AI Use By Five Federal Agencies In A New Report, via Buzzfeed News.

#NHS

Oesophageal cancer is one of the deadliest forms of the disease because of how difficult it is to diagnose. So experts from several universities and a technology firm teamed up to develop CADU, a software that uses artificial intelligence to identify cancerous cells in the food pipe. The device was trained on thousands of images of diseased tissue and learned to spot cancerous growths based on the patterns. It was approved by the Medicines and Healthcare Products Regulatory Agency to improve its diagnosis rates and is now in use at University College London hospital trust.

AI software may help spot early signs of oesophageal cancer, via The Guardian.

** Per Wikipedia, researchers at the University of Plymouth actually tried this with a bunch of monkeys for a month back in 2002. “Not only did the monkeys produce nothing but five total pages largely consisting of the letter ‘S’, the lead male began striking the keyboard with a stone, and other monkeys followed by soiling it.”

____

📄 Paper of the Week

GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields

This paper won the best paper award at CVPR this year and presents a novel combination of some popular ideas: GANs and Neural Volume Rendering. Controllable image generation with GANs is usually framed into separating the latent space such that dimensions correspond to useful parameters, e.g. pose. This paper turns away from that branch of thinking and instead composes GAN-generated images by using explicit poses (provided by the human). These sub-images are then combined into a full image using a type of volume rendering. For those in the 3D world, this type of explicit scene composition is exactly what we do already, except we use meshes instead of 2D GAN-generated images. This work represents a middle ground between the GAN world and the 3D world and is likely the beginning of a true deep learning-based, fully controllable scene generation.

____

Think The Juice was worth the squeeze? Sign up here to receive it weekly.