May 2, 2024
Viren Jain, Research Scientist and Technical Lead, Connectomics at Google
Researchers built a 3D image of nearly every neuron and their connections within a small piece of human brain tissue. The left [top] image shows excitatory neurons and the right [bottom] image shows inhibitory neurons. These versions are shaded according to the size of the neurons’ cell bodies (central core), which range from 15–30 micrometers across. The sample is approximately 3 mm wide. Credit: Google Research & Lichtman Lab (Harvard University). Renderings by D. Berger (Harvard University).
The human brain uses no more power than a dim incandescent light bulb, yet it can accomplish feats still not possible with the largest artificial computing systems. To understand how requires a level of understanding more profound than knowing what part of the brain is responsible for what function. The field of connectomics aims to achieve this by precisely mapping how each cell is connected to others. Building detailed maps of how brains from many types of organisms are wired is transforming our understanding of how brains work. This could help researchers understand neurological disorders and also answer fundamental questions, such as how memories form.
This year marks the tenth anniversary of the formation of Google Research’s Connectomics research team. We have made connectomes possible through the development of machine learning (ML) algorithms and software tools that process and visualize data at unprecedented scale. Marking this anniversary, today we publish in Science, “A petavoxel fragment of human cerebral cortex reconstructed at nanoscale resolution” in collaboration with Jeff Lichtman of Harvard University and others. Released previously as a preprint, this connectome describes a piece of human brain tissue the size of half a grain of rice that nonetheless requires 1.4 petabytes (1.4 million gigabytes) to encode — including about 16,000 neurons, 32,000 glia, 8,000 blood vessel cells (for a total of ~57,000 cells) and 150 million synapses. This project revealed never-before-seen structures within the human brain that may change our understanding of how our brains work.
Since we published the preprint, we have expanded our suite of interactive, open source tools that enable researchers to investigate the dataset on their own. The ability for other researchers to proofread and refine this human brain connectome is one of many ways that we see the release of this paper and the associated tools as not only the culmination of 10 years of work, but the beginning of something new.
The first connectome was published in 1986 — before AI tools existed — for the 302 neurons in the nematode model organism Caenorhabditis elegans. It took 16 years to create it from cross-sectional microscope images of the worm. Researchers manually colored in cells from one cross-section to another to visualize the connections in this simple nervous system.
When the Connectomics team at Google launched ten years ago, we were excited about how innovations in AI and working with large datasets could enable us to move from 302 neurons to the tens of thousands or millions found in more complex organisms. Our work required novel algorithms capable of handling the tremendous amounts of data — now petabytes — these studies generate. We developed flood-filling networks to replace the manual effort of coloring in cells across images. These networks allow automated reconstruction of neurons through layers of tissue. Building on this, our SegCLR algorithm automatically identifies distinct parts of cells and cell types within these networks. We also developed TensorStore, an open-source C++ and Python software library to store and manage massive multi-dimensional datasets. This tool has realized benefits well beyond connectomics and is now widely used at Google and across the broader ML community.
We first demonstrated these algorithms when we released the connectome for the “hemibrain” of the fruit fly Drosophilamelanogaster in 2020. Revealing the connections among 25,000 neurons in a central portion of the fruit fly brain, this reconstruction has been used by other researchers to make findings about learning, memory, and behavior in the fruit fly. Groups have since published hundreds of papers that build on the fruit fly connectome.
Through collaborations with researchers at the Howard Hughes Medical Institute, Harvard University, and Max Planck Institute we have also published connectomes for portions of the brains of the zebra finch and zebra fish larvae.
In the work published today, our team reports a new milestone: a synaptic-resolution reconstruction of a 1 cubic mm piece of human brain tissue. Our collaborators generated the dataset using a sample of brain tissue from the left anterior temporal lobe that was removed during brain surgery on a person with epilepsy. Lichtman’s team used a multibeam scanning electron microscope to gather high resolution images of more than 5,000 slices of tissue, each roughly 30 nanometers thick. Image acquisition alone took 326 days. Then our team’s tools stitched and aligned the image data, reconstructed the three dimensional structure of each cell, including its axons and dendrites, identified synaptic connections, and classified cell types. The reconstruction revealed several surprises.
The brain tissue sample used in this research came from the left anterior temporal lobe.
For example, we found a class of rare but extremely powerful synaptic connections in which a pair of neurons may be connected by more than 50 individual synapses. While 96.5% of contacts between axons and their target cells have just one synapse, 0.092% have four or more synaptic connections. The conformation of these connections combined with refined statistical analysis that is part of the Science publication suggests that these powerful connections are not the result of chance, but rather that these pairs had a reason to be more strongly connected than is typical. Further study of these connections could reveal their functional role in the brain. Perhaps, for example, these strong connections are how the brain achieves particularly fast neural responses or how it encodes very important memories. […]
In very rare cases, a single axon (blue) made repeated synaptic connections (yellow) with a target neuron (green). The purpose of these very tight connections is not yet known. Credit: Google Research & Lichtman Lab (Harvard University). Renderings by D. Berger (Harvard University).
Full article: Ten years of neuroscience at Google yields maps of human brain
September 18, 2024
Lauren Harrell, Data Scientist, Google Research
We introduce our new whale bioacoustics model, which can identify eight distinct species, including multiple calls for two of those species. The model also includes the “Biotwang” sounds recently attributed to the Bryde’s whale.
In order to protect animals that live in remote environments, researchers must be able to find them to understand the movements of their populations over time. As long-term passive acoustic monitoring capabilities have grown more technologically sophisticated, automatic animal species identification tools built on large datasets from these recorded soundscapes have become an increasingly vital tool for conservation and ecological research. While models such as Google Perch have emerged that can classify thousands of bird vocalizations, similar models that can classify vocalizations from several whale species at once have proven more challenging to develop.
The acoustic range of whale species is incredibly broad, ranging from as low as 10 Hz for blue whales to above 120kHz for odontocetes (toothed whales), and recordings also vary dramatically by location and with time, which can make model development difficult. Additionally, researchers often don’t know what types of vocalizations are made by some especially elusive whale species, which complicates identifying those animals in the soundscapes. This is illustrated in the decades-old mystery surrounding a sound, called a “Biotwang”, that was first recorded almost a decade ago in the depths of the Mariana Trench. The sound has a “metallic” or “chime-like” quality, quite unlike the tonal moans more typical of whale vocalizations. In a recent paper, our collaborators at the U.S. National Oceanic and Atmospheric Administration (NOAA) determined that the Biotwang sound is uniquely produced by the elusive Bryde’s whales (pronounced “broodus”).
Today we are delighted to share Google’s latest whale bioacoustics model, which can identify eight distinct species, as well as multiple calls for two of those species. Following on our collaborator’s discovery connecting Biotwangs to the Bryde’s whale and in the same paper, we expanded the model to include Biotwangs and used it to label more than 200,000 hours of underwater recordings. Here we describe the model and discuss some of the new insights into the ecology of whale species it is helping researchers to unlock. The model is now available for download via Kaggle Models.
Google Research’s journey with whale vocalization classification started in 2018 when we developed a novel classification model for detecting humpback whales in partnership with the Pacific Islands Fisheries Science Center (PIFSC) of NOAA. The model was used to identify humpback calls from over 187,000 hours of audio collected by NOAA, confirming spatio-temporal patterns of humpback songs and uncovering a new location at Kingman Reef where humpback songs had not been previously observed. We made further “splashes” with this model in collaboration with Google Creative Lab when we released Pattern Radio, an interactive visualization of a full year of underwater audio collected near Hawaii, labeled by the model, and peppered with additional expert insights on sections of the data. We released our humpback model publicly following Google’s AI Principles to understand and minimize the potential for misuse of the model.
These efforts led to a partnership with the Department of Fisheries and Oceans, Canada (DFO), especially with their Marine Mammal Response Program, whose operations in the Salish Sea include stewardship of the critically endangered Southern Resident Killer Whale population. Together, we published an orca (killer whale) detection model, which DFO also deployed in their hydrophone monitoring network, enabling real-time alerts.
We developed our new multi-species whale model to score and classify underwater audio for eight distinct species. Two of the species are further broken down by vocalization type, yielding a total of twelve classes. The model is multi-label, so scores are independent and not restricted to the top class or classes.
The following is the list of species for which the model can provide classification scores:
Audio examples of each of the species are in this repository.
[…]
Bryde’s whales are baleen whales in the same genus as blue and fin whales. While sightings of these animals have been reported around the world, relatively little is known about their movements or population structure. Recordings collected in the Mariana Trench in 2014 and 2015 captured a unique vocalization called the “Biotwang”. This complex, 5-part call lasts approximately 3.5 seconds, starting with a low-frequency downsweeping moan from approximately 44 Hz to 30 Hz and followed by a “metallic” sound that goes up to 8000 Hz. Because the vocalizations were not associated with sightings of the animals, the researchers originally attributed them to an undetermined baleen species.
Photo of a Bryde’s whale spotted during a NOAA cetacean survey in the Mariana Archipelago in 2010 (Credit: NOAA Fisheries/Adam Ü, NMFS MMPA-ESA Permit #14097).
Subsequently, NOAA researchers were able to attribute Biotwangs to Bryde’s whales by aligning visual observations and acoustic data captured by sonobuoys. In the new paper, they report the true identity of these long-mysterious, twangy calls for the first time. That positive identification enabled us to improve our multi-species whale model by labeling the Biotwangs as Bryde’s whale signatures in the training data. When applied to the collection of long-term passive acoustic datasets, this led to the discovery of many instances of this call in the western North Pacific Ocean, revealing potential population differences between the central and western Pacific Bryde’s whales and uncovering a seasonality to these whales’ migration patterns.
Audio and spectrogram of the Biotwang call of a Bryde’s whale. The highest intensity downsweep is at low frequencies, while the additional lines directly above are harmonics that aren’t always present. The part that looks like quotation marks is the higher frequency “metallic” sound.Full article: Whistles, songs, boings, and biotwangs: Recognizing whale vocalizations with AI
August 5, 2024
Art
Craft
Grace Ebert
Two spirited lambs bound across a new wooly pasture hand-felted by Philadelphia-based artist Holly Guertin.
Two spirited lambs bound across a new wooly work by Philadelphia-based artist Holly Guertin. Inspired by a spring visit to a local farm, Guertin felted a pair of sprightly youngsters with striking detail. Pink lines their ears and noses, and their adolescent bodies are covered in tight, wavy fibers, all sculpted through a combination of wet and needle felting techniques.
Titled “The Invitation,” the soft, fleecy work stretches approximately four feet on each side. Guertin shares with Colossal that she’s interested in the relationship between sheep and their wool and whether humans, like sheep, have something similarly tangible to offer.[…]
More: A Pair of Fluffy Felted Lambs by Holly Guertin Spring to Life in a Wooly Pasture
This video features images of the largest and most complex brain ever fully mapped by scientists – that of an adult fruit fly. To bring these detailed images to life, it took scientists from 146 labs and 122 institutions in a project known as FlyWire, led by Princeton University. If charting the brain of this small creature sounds like anything less than an extraordinary breakthrough, consider that the 140,000 neurons and the many millions of synapses the project details mark an extraordinary leap from the worm brain (302 neurons) and the larval fruit fly brain (3,000 neurons) that scientists have previously mapped. And, while fully mapping a human brainof roughly 86 billion neurons is likely still many years from fruition, the team behind FlyWire believes that their project could represent a formidable step towards better understanding brain diseases such as dementia, Alzheimer’s and Parkinson’s.
Via Colossal
Video by FlyWire Princeton
Source: This intricate map of a fruit fly brain could signal a revolution in neuroscience | Aeon Videos
Hosted by Elvis Mitchell Sep. 28, 2024
Kate Winslet. Photo credit: Alexi Lubomirski
Emmy and Oscar-winner Kate Winslet joins for a long conversation about her Lee Miller biopic, portraying PTSD, and continuing to challenge herself as an actor.
Kate Winslet is considered one of the greatest actresses of her generation — an Oscar win for her role in 2008’s The Reader and Emmy wins for leading roles in both Mare of Easttown and Mildred Pierce bear this out. Her range as a performer is boundless with a resumé boasting period dramas like Sense and Sensibility, edgy comedies like Eternal Sunshine of the Spotless Mind, and (of course) one of the biggest films of all time: Titanic.
Her latest endeavor finds her producing and starring in the film Lee. The ambitious project centers Lee Miller, the groundbreaking American photojournalist who covered World War II for Vogue magazine. In an extended and candid conversation, Winslet tells The Treatment about the struggles she endured while getting the film made. She talks about portraying several women within recent years who all dealt with some form of PTSD. And she shares why she keeps looking for ways to get better as an actor.
Kate Winslet and Elvis Mitchell at KCRW. Photo credit: Rebecca Mooney.
Listen: Kate Winslet’s candid and expansive chat with Elvis Mitchell
