Hi guys and welcome to TechCrunch’s regular AI newsletter.
This week, Apple was in the spotlight in the AI sector.
At the Worldwide Developers Conference (WWDC) in Cupertino, Apple unveiled Apple Intelligence, its long-awaited, ecosystem-wide foray into generative AI. Apple Intelligence enables a whole host of features, from an improved Siri to AI-generated emojis to photo editing tools that remove unwanted people and objects from photos.
The company promised that Apple Intelligence would be developed with security at its core and enable highly personalized experiences.
“It needs to understand you and be anchored in your personal context, like your daily routine, your relationships, your communications and more,” CEO Tim Cook noted during Monday’s keynote. “All of this goes beyond artificial intelligence. It’s personal intelligence and the next big step for Apple.”
Apple Intelligence is typical Apple: It hides the technical details behind obvious, intuitively useful features. (Cook never once uttered the phrase “big language model.”) But as someone who writes about the dark side of AI for a living, I wish Apple would be more transparent—just this once—about how the sausage is made.
Take Apple’s model training practices, for example. Apple disclosed in a blog post that it trains the AI models that power Apple Intelligence using a combination of licensed datasets and the public web. Publishers have the option to opt out of future training. But what if you’re an artist who wants to know if your work was included in Apple’s initial training? Tough luck—there’s no word on that.
The secrecy could be for competitive reasons. But I suspect it’s also to protect Apple from legal challenges—particularly challenges related to copyright. Courts have yet to decide whether vendors like Apple have the right to train on public data without compensating or naming the creators of that data—in other words, whether the fair use doctrine applies to generative AI.
It’s a little disappointing to see Apple, which often presents itself as a champion of sensible technology policy, implicitly support the fair use argument. Behind the veil of marketing, Apple can claim to pursue a responsible and measured approach to AI, while it may well have trained on developers’ works without permission.
A little explanation would go a long way. It’s a shame we didn’t get one – and I’m not confident we’ll get one any time soon unless a lawsuit (or two) comes along.
News
Apple’s key AI features: Your humble servant has summarized the key AI features Apple announced this week during the WWDC keynote, from improved Siri to deep integrations with OpenAI’s ChatGPT.
OpenAI is hiring executives: OpenAI this week hired Sarah Friar, the former CEO of hyperlocal social network Nextdoor, as chief financial officer and Kevin Weil, who previously led product development at Instagram and Twitter, as chief product officer.
Mail, now with more AI: This week, Yahoo (TechCrunch’s parent company) updated Yahoo Mail with new AI features, including AI-generated email summaries. Google recently launched a similar generative summarization feature – but it’s behind a paywall.
Controversial views: A recent study from Carnegie Mellon University finds that not all generative AI models are the same – especially when it comes to dealing with polarizing issues.
Sound generator: Stability AI, the startup behind AI-powered art generator Stable Diffusion, has released an open AI model for generating sounds and songs that it says was trained exclusively on royalty-free recordings.
Research paper of the week
Google believes it can create a generative AI model for personal health – or at least take initial steps in that direction.
In a new article published on the official Google AI blog, researchers at Google lift the curtain on the Personal Health Large Language Model, or PH-LLM for short – a refined version of one of Google’s Gemini models. PH-LLM is designed to provide recommendations to improve sleep and fitness, including by reading heart and respiratory rate data from wearable devices such as smartwatches.
To test the ability of PH-LLM to provide useful health recommendations, researchers conducted nearly 900 case studies on sleep and fitness with U.S. subjects. They found that PH-LLM provided sleep recommendations that close to – but not quite as good as – the recommendations of sleep experts.
The researchers say PH-LLM could help contextualize physiological data for “personal health applications.” Google Fit springs to mind; I wouldn’t be surprised if PH-LLM eventually powers a new feature in a fitness-focused Google app, whether Fit or something else.
Model of the week
Apple has written quite a bit on its blog about the new on-device and cloud-based generative AI models that make up the Apple Intelligence Suite. However, despite the length of this post, it reveals very little about the capabilities of the models. Here is our attempt to analyze it:
The unnamed on-device model that Apple is highlighting is small, no doubt so it can run offline on Apple devices like the iPhone 15 Pro and Pro Max. It contains 3 billion parameters — “parameters” are the parts of the model that essentially define its ability to do a problem, like generating text — making it comparable to Google’s on-device Gemini model, Gemini Nano, which comes in 1.8 billion parameter and 3.25 billion parameter sizes.
The server model is now larger (Apple does not say exactly how much larger). What we Do What is known is that it is more powerful than the on-device model. While the on-device model performs on par with models like Microsoft’s Phi-3 mini, Mistral’s Mistral 7B and Google’s Gemma 7B in the benchmarks listed by Apple, the server model “compares well with OpenAI’s older flagship model, GPT-3.5 Turbo,” according to Apple.
Apple also says that both the on-device model and the server model are less prone to getting out of hand (i.e., spewing toxicity) than similarly sized models. That may be so—but this author reserves judgment until we have a chance to put Apple Intelligence to the test.
Grab bag
This week marked the sixth anniversary of the release of GPT-1, the predecessor to GPT-4o, OpenAI’s latest flagship model for generative AI. And while deep learning is currently reaching its limits, it’s incredible how far the field has come.
Consider that it took a month to train GPT-1 on a dataset of 4.5 gigabytes of text (BookCorpus, which contains about 7,000 unpublished novels). GPT-3, which is almost 1,500 times larger than GPT-1 by parameter count and significantly more sophisticated in the prose it can generate and analyze, took 34 days to train. How does that scale?
What made GPT-1 groundbreaking was its training approach. Previous techniques relied on huge amounts of manually labeled data, which limited their usefulness. (Manually labeling data is time-consuming—and tedious.) But GPT-1 did not do this; it trained primarily on unlabeled Data to “learn” how to perform a series of tasks (e.g., writing essays).
Many experts believe that we will not see a paradigm shift as significant as that of GPT-1 in the near future. But then again, the world did not see GPT-1 coming either.