Why Apple is Taking a Small-model Approach to Generative AI | TechCrunch - Latest Global News

Why Apple is Taking a Small-model Approach to Generative AI | TechCrunch

One of the biggest questions that has arisen since the launch of models like ChatGPT, Gemini and Midjourney is what role (if any) they will play in our daily lives. Apple is trying to answer that question with its own version of the category, Apple Intelligence, which was officially unveiled this week at WWDC 2024.

The company dazzled with highlights at Monday’s presentation; that’s just how keynotes work. When SVP Craig Federighi wasn’t doing some Hollywood (or Cupertino) magic to demonstrate skydiving or parkour, Apple was determined to show that its own models are every bit as good as the competition.

The question is still up in the air, as beta versions only came out on Monday, but the company has since revealed some of the specifics of its approach to generative AI. First and foremost, it’s about scale. Many of the best-known companies in the space take a “bigger is better” approach to their models. The goal of these systems is to serve as a kind of one-stop shop for the world’s information.

Apple’s approach in this category, however, is based on something more pragmatic. Apple Intelligence is a bespoke approach to generative AI, tailored specifically to the company’s various operating systems. It’s a very Apple approach in the sense that it puts a frictionless user experience first and foremost.

On the one hand, Apple Intelligence is a branding exercise, but on the other, the company prefers to integrate the generative AI aspects seamlessly into the operating system. It’s perfectly fine – or even desirable – if the user has no idea about the underlying technologies that power these systems. This is how Apple products have always worked.

Keep the models small

The key to much of this is building smaller models: the systems are trained on a bespoke dataset specifically tailored to the kind of functionality that users of their operating systems need. It’s not immediately clear how much the size of these models will affect the black box problem, but Apple believes that at least more topic-specific models will increase transparency about why the system makes certain decisions.

Due to the relatively limited nature of these models, Apple does not expect a great deal of variety when asked to, for example, summarize text. Ultimately, however, the variation from prompt to prompt will depend on the length of the text being summarized. The operating systems also have a feedback mechanism through which users can report problems with the generative AI system.

Although Apple Intelligence is much more focused than larger models, it can cover a wide range of needs thanks to the inclusion of “adapters” that specialize in different tasks and styles. In general, however, Apple doesn’t take a “bigger is better” approach to creating models, as things like size, speed, and processing power must be taken into account – especially when working with device models.

ChatGPT, Gemini and the rest

Given the limited focus of Apple’s models, it makes sense to open up to third-party models like OpenAI’s ChatGPT. The company has trained its systems specifically for the macOS/iOS experience, so there will be a lot of information that is outside their scope. In cases where the system thinks a third-party application would be better suited to provide an answer, a system query will ask if you want to share that information externally. If you don’t get such a query, the request will be processed using Apple’s internal models.

This should work the same way for all external models Apple works with, including Google Gemini. This is one of the rare cases where the system alerts to the use of generative AI in this way. The decision was made in part to address privacy concerns. Each company has different standards when it comes to collecting and training user data.

Having users opt in each time removes some of the responsibility from Apple, even if it makes the process a little more difficult. You can also opt out of using third-party platforms system-wide, but that would limit the amount of data the OS/Siri can access. However, you can’t opt ​​out of Apple Intelligence in one fell swoop. Instead, you have to do it for each feature individually.

Private Cloud Computing

However, it is not made clear whether the system processes a particular query on the device or via a remote server using private cloud compute. Apple’s philosophy is that such disclosures are not necessary because the company applies the same data protection standards to its servers as it does to its devices, right down to the first-party chip they run on.

To know for sure if the query is running on the device or off, disconnect your computer from the Internet. If the problem requires cloud computing but the computer can’t find a network, it will throw an error indicating that the requested action cannot be performed.

Apple explains the specifics of what actions require cloud-based processing. Several factors come into play, and the ever-changing nature of these systems means that something that requires cloud computing today may be able to be done on-device tomorrow. On-device computing will not always be the faster option, as speed is one of the parameters Apple Intelligence considers when determining where to process the prompt.

However, there are certain operations that always run on the device. The most notable of these is Image Playground, as the complete diffusion model is stored locally. Apple has optimized the model to generate images in three different house styles: animation, illustration, and sketch. The animation style is very similar to the house style of another company founded by Steve Jobs. Likewise, text generation is currently available in a trio of styles: friendly, professional, and concise.

Even in this early beta stage, Image Playground’s generation is impressively fast, often taking just a few seconds. As for the question of inclusion when generating people’s images, the system requires you to enter details rather than simply guessing things like ethnicity.

How Apple will handle data sets

Apple’s models are trained using a combination of licensed datasets and by crawling publicly available information. The latter is achieved using AppleBot. The company’s web crawler has been available for some time and provides contextual data for applications such as Spotlight, Siri and Safari. The crawler has an existing opt-out feature for publishers.

“With Applebot-Extended,” Apple says, “web publishers can opt out of having their website content used to train Apple’s foundational models that power generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.”

This is achieved by embedding a prompt in the website’s code. With the launch of Apple Intelligence, the company introduced a second prompt that allows websites to be included in search results but excluded from training generative AI models.

Responsible AI

Apple released a white paper on the first day of WWDC titled “Introducing Apple’s On-Device and Server Foundation Models.” Among other things, it highlights the principles of the company’s AI models. In particular, Apple highlights four things:

  1. “Empower users with intelligent tools: We identify areas where AI can be used responsibly to create tools that meet specific user needs. We respect how our users use these tools to achieve their goals.”
  2. “Represent our users: We build deeply personal products with the goal of authentically representing users around the world. We continuously work to avoid perpetuating stereotypes and systemic biases in our AI tools and models.”
  3. “Design carefully: We take precautions at every stage of our process, including design, model training, feature development, and quality assessment, to identify how our AI tools could be misused or lead to potential harm. We will continuously and proactively improve our AI tools using user feedback.”
  4. “Protect privacy: We protect our users’ privacy through powerful on-device processing and groundbreaking infrastructure such as private cloud compute. We do not use our users’ private personal data or user interactions when training our base models.”

Apple’s bespoke approach to basic models allows the system to be tailored specifically to the user experience. The company has taken this UX-first approach since the launch of the first Mac. Providing the smoothest experience possible benefits the user, but it shouldn’t come at the expense of privacy.

That will be a difficult balance the company will have to master when the current generation of beta versions of the operating system becomes generally available this year. The ideal approach is to offer as much – or as little – information as the end user needs. Certainly there will be many people who don’t care whether a query is run on the computer or in the cloud, for example. They are happy for the system to default to whatever is most accurate and efficient.

For privacy advocates and others interested in these details, Apple should aim for as much transparency as possible for users – not to mention transparency for publishers who may prefer not to have their content used as a source for training these models. There are certain aspects where the black box issue is currently unavoidable, but in cases where transparency can be provided, it should be provided upon user request.

Sharing Is Caring:

Leave a Comment