Explore. Build. Grow
Explore. Build. Grow

Our flexible three-stage framework to craft digital experiences.

Learn more
Expertise
Outcomes
View our portfolio
View our portfolio

Explore our range of digital solutions.

Learn more
Industries
Optimizely
Optimizely

Kentico
Kentico

Umbraco
Umbraco

DXP
Headless
Cloud Platforms
Marketing Optimisation
Contact Us
Contact Us
Explore. Build. Grow
Explore. Build. Grow

Our flexible three-stage framework to craft digital experiences.

Learn more
Expertise
Outcomes
View our portfolio
View our portfolio

Explore our range of digital solutions.

Learn more
Industries
Optimizely
Optimizely

Kentico
Kentico

Umbraco
Umbraco

DXP
Headless
Cloud Platforms
Marketing Optimisation
Contact Us
Manohar Paluri with Ryan Patel talking at SXSW Sydney 2024.
Home / Insights / ＿

The future of AI: open source and democratisation with Meta

Meta's decision to open-source its AI LLM project, Llama 3.2, is a significant step towards responsible and collaborative AI development, argues QA Analyst Iqbal Tawakkal.

Iqbal Tawakkal
Iqbal Tawakkal

01 December 2024

6 minute read

The field of artificial intelligence is rapidly evolving. Many AI Large Language Models (LLMs), such as OpenAI's models, Claude AI with its Sonnet framework, and Google's Gemini, are developed in closed-source environments. (Notably, OpenAI has not released its source code despite having 'Open' in its name). 

Fortunately, major companies like Meta are contributing to open-source AI development with projects such as Llama 3.2. 

At SXSW, I had the opportunity to hear directly from Manohar Paluri, the VP of AI at Meta. He is leading the initiative to develop and democratise AI at the company.

Keeping Llama open source

Meta's recent release of Llama 3.2 marks a significant advancement in open-source AI research. By making these powerful models accessible to the community, Meta underscores its belief in the importance of open access for unlocking AI's full potential. This approach empowers academics, developers and innovators to freely create, test and publish AI applications across various domains, fostering a thriving environment for innovation and the emergence of groundbreaking applications.

The decisions we make today regarding AI accessibility will have profound implications for the future of technology. 

Llama 3.2's open-source nature plays a pivotal role in democratising access to AI technology. As AI becomes increasingly integrated into our daily lives, it is crucial to ensure that its benefits are accessible to all. Open-source projects like Llama 3.2 break down barriers to entry, enabling a wider range of individuals and organisations to participate in AI development and application.

Furthermore, open-source initiatives promote transparency and collaboration within the AI community. These values are of paramount importance as AI's influence continues to expand. By fostering an environment of openness and collaboration, Llama 3.2 and similar projects contribute to the responsible and ethical development of AI technologies.

Notable features of Llama 3.2

Llama 3.2's capabilities are supported by a range of impressive features that distinguish it:

  • Enhanced Model Architecture: Llama 3.2 introduces a refined architecture featuring innovations like 'grouped query attention'. This mechanism allows the model to prioritise and focus on the most relevant parts of the input text, resulting in more accurate and contextually appropriate responses.
  • Flexibility and control: As an open-source model, Llama 3.2 offers unparalleled control, allowing developers to fine-tune, integrate and create new applications easily and allowing for much faster development in AI.
  • Multilingual and multimodal capabilities: Llama 3.2 supports multiple languages and multimodal inputs. Its training incorporates a diverse linguistic foundation, including over five percent high-quality non-English text spanning more than 30 languages. While this broadens the model’s language capabilities, English remains the primary language and also its performance is somewhat limited due to less availability of multilingual data. It’s sometimes biased towards English due to a larger volume of English data in its training set, resulting in stronger proficiency and accuracy in English compared to other languages.
  • Extensive and diverse training data: The power of any LLM hinges on the quality and diversity of its training data. Meta has provided Llama 3.2 with a massive dataset encompassing text and code in multiple languages, ensuring that the model is not limited to English but can effectively handle multilingual tasks and understand cultural nuances.
  • Benchmark performance: Meta states that Llama 3.2 achieves exceptional results on industry-standard benchmarks designed to evaluate LLM performance. These benchmarks assess a model's proficiency in various tasks, including question answering, text summarisation, and creative writing. While specific details are yet to be fully disclosed, initial indications suggest that Llama 3.2 can compete favourably with, and potentially even surpass, the performance of established models like GPT-4.

The necessity of democratising AI development

The decisions we make today regarding AI accessibility will have profound implications for the future of technology. Open-source AI represents a paradigm shift, emphasising principles that are essential for shaping a responsible and inclusive AI landscape:

  • Transparency: Open-source AI fosters transparency by allowing researchers and developers to scrutinise the model's inner workings. This transparency is vital for understanding how AI systems make decisions and ensuring that they are free from bias and discrimination.
  • Accountability: Community-driven development and oversight of open-source AI models promote accountability. By enabling a diverse group of stakeholders to participate in the development process, we can collectively identify and address potential issues, ensuring that AI technologies are used ethically and responsibly.
  • Innovation: Open-source AI encourages rapid iteration and innovation. By making AI models and tools freely available, we empower a global community of researchers and developers to collaborate, experiment, and build upon each other's work, accelerating the pace of progress.
  • Education: Open-source AI provides accessible learning resources for aspiring AI developers. By democratising access to cutting-edge AI technologies, we can nurture a new generation of talent and ensure that the benefits of AI are widely shared.
  • Ethics: Open-source AI promotes democratic participation in shaping the future of AI. By enabling a broad range of voices to contribute to the development and governance of AI systems, we can ensure that these technologies align with our shared values and serve the common good.

Open-sourcing Llama 3.2: Driving innovation and collaboration

Meta's decision to keep Llama open-source is rooted in the belief that collaboration is key to responsible and rapid AI development. Here's why:

  • Accelerated innovation: By granting access to researchers and developers globally, Llama 3.2's capabilities can be rapidly enhanced and expanded upon. In 2012, the best AI model required 10-20 times more computational power than current models.
  • Strengthened safety and security: The open-source model invites community scrutiny, enabling faster identification and mitigation of potential risks associated with large language models (LLMs).
  • A flourishing AI ecosystem: Open access fosters a competitive yet collaborative environment. This dynamic drives the creation of superior LLMs, innovative applications, and ultimately delivers greater benefits to society.

With great power comes great responsibility

Llama 3.2 signifies a substantial leap forward in AI capabilities, but its true distinction lies in its dedication to ethical behaviour and safety. The proactive measures taken to address potential biases and inaccuracies demonstrate a commitment to responsible AI development. This focus on ethics is not just about mitigating harm; it's about building trust. It's about ensuring that AI technology serves as a force for good, rather than a source of societal problems.

The development of Llama Guard models further underscores this commitment to safety. These models act as a safeguard, helping to identify and mitigate potential risks associated with AI use. This proactive approach to risk management is essential for ensuring that AI technology is used in a way that benefits society as a whole. It's a clear indication that the creators of Llama 3.2 are not just focused on advancing AI capabilities; they're equally committed to ensuring that these capabilities are used responsibly.

The question then becomes: how can other AI developers follow this example? How can we ensure that the pursuit of AI advancement is always tempered by a commitment to ethics and safety? The Llama 3.2 project provides a compelling blueprint, but it's just the beginning. As AI continues to evolve, it's imperative that we all work together to ensure that it's used in a way that benefits humanity, rather than harms it.

It’s like having a friendly but vigilant teacher who ensures Llama 3.2 plays by the rules and doesn’t cause any harm.

Meta: The unexpected good guy?

Hold onto your hats, folks, because Meta might just be the only big tech giant playing it straight with AI! Llama 3.2 might not be setting the world on fire, but it's a solid start, especially since they're kinda, sorta making it open source. Sure, there might be some sneaky motives lurking in the shadows, but for now, it's a big win for developers. Could Meta be trying to polish up their image? Who knows, but let's enjoy the ride!

Iqbal Tawakkal
Iqbal Tawakkal

01 December 2024

6 minute read

Enjoyed this article? Share it...

Keep Reading

Want more? Here are some other blog posts you might be interested in.

Follow
Contact Us
1300 554 891
[email protected]

Our locations

Stay informed
Our best articles, observations and news delivered to your inbox
Subscribe
Flags

We acknowledge the Traditional Custodians of the land on which we work, live and engage. We pay respect to Elders past and present.

Luminary is a registered trademark of Luminary Digital Pty Ltd

Terms of Services
Privacy Policy