Month / September 2024

by Thomas

9 AI trends in Late 2024

From smaller, more efficient language models to groundbreaking multi-modal capabilities, AI is becoming increasingly accessible and versatile. In this blog post, we’ll look at nine key AI trends that are shaping the landscape of artificial intelligence.

Reality Check

The initial hype surrounding AI in 2023 has given way to a more realistic assessment of its capabilities and limitations. While AI has made significant progress, challenges such as bias and ethical concerns remain. As researchers continue to refine AI models and address these issues, we can expect further advancements in various fields. However, it’s crucial to maintain a balanced perspective and recognize that AI is a tool that, when used responsibly, can benefit society.

Multi-Modal AI

AI is making significant strides with multi-modal models. These innovative systems can process and understand information from various sources, including text, images, audio, and video. This opens up a world of possibilities, such as generating captions for images or creating videos based on audio. Multi-modal AI is revolutionizing fields like medical image analysis and customer service, making machines more intelligent and versatile than ever before.

Smaller Models

Forget “bigger is better,” 2024 is seeing a major shift in AI with the rise of smaller language models (LLMs). This move away from giant, complex models offers a wave of benefits. Smaller LLMs are more energy-efficient and cost-effective, making them accessible for businesses and individuals alike. They can even run on personal computers and smartphones, putting AI power in everyone’s hands. Plus, smaller models often focus on specific tasks, leading to more focused and efficient solutions – whether it’s crafting compelling content or streamlining customer service. This trend towards smaller LLMs reflects a future where AI becomes more accessible, efficient, and specialized than ever before.

Custom Local Models

Custom local models are gaining prominence in AI. These models, trained on private datasets within an organization’s infrastructure, offer a tailored approach to AI applications. By maintaining control over sensitive information, organizations can enhance data privacy and comply with regulations. Custom models can also be fine-tuned for specific tasks, improving accuracy and relevance.

While training custom models can be computationally expensive, fine-tuning is cheaper and the benefits often outweigh the costs. For industries like healthcare, finance, and manufacturing, these models offer significant advantages. By operating offline, they enable applications in environments with limited or no internet connectivity. Additionally, reduced latency improves user experience and response times. To address the challenges of training and maintenance, organizations can leverage transfer learning techniques and invest in appropriate hardware resources.

GPU + Cloud Costs

GPU and cloud costs continue to be major factors in AI development. As AI models, especially LLMs, grow more complex, the demand for powerful hardware and scalable cloud infrastructure increases, driving up costs. High-performance GPUs remain expensive, while cloud providers offer various GPU options at different prices. To manage costs, AI practitioners use techniques like model compression and distributed training. Organizations must carefully weigh the costs of GPUs against the benefits of cloud computing, considering factors like project scale and desired performance. By effectively managing these costs, businesses can maximize the value of their AI investments.

Model Optimization

In 2024, model optimization is a key trend in AI, focused on making models smaller, faster, and more adaptable without sacrificing performance. Techniques like LoRA (Low-Rank Adaptation) allow for efficient fine-tuning of large models by reducing the number of parameters to be updated, while quantization lowers model precision to reduce size and improve speed, especially on edge devices. Pruning further trims down models by removing unnecessary weights, and knowledge distillation transfers the capabilities of large models to smaller ones, maintaining high performance. These methods, alongside sparse models and memory-efficient attention mechanisms, are helping make AI more scalable and accessible. With hardware-aware optimization and dynamic scaling, AI models are increasingly efficient, capable of running on a wider range of devices, from cloud servers to mobile platforms.

Virtual Agents

Virtual agents are becoming increasingly sophisticated, automating tasks and providing personalized assistance across various industries. From scheduling meetings to managing personal finances, these AI-powered assistants are streamlining workflows and saving time for users. As AI technology continues to advance, we can expect virtual agents to play an even more significant role in our daily lives.

Shadow AI

Shadow AI, a growing concern in 2024, refers to the unauthorized use of AI technologies within an organization. It’s like the shadowy underbelly of the digital world, where employees secretly adopt AI tools without IT’s blessing.

This trend is fueled by the increasing accessibility and power of AI. Employees often turn to Shadow AI for quick solutions, cost-effectiveness, or personal preference. However, this seemingly harmless behavior can lead to serious consequences.

Regulation

The world of AI regulation is undergoing a major shift. 2024 has seen a surge in legislation, with the EU’s AI Act setting a framework for risk-based regulation. Countries are also introducing their own rules focused on data privacy, transparency,and fairness in AI development. Regulatory bodies are demanding accountability for potential bias in AI systems, leading to an increase in bias audits by organizations. Meanwhile, research into AI safety is growing, with international cooperation aimed at developing responsible AI standards. The key challenge lies in balancing innovation with ethical development, while striving for global harmonization to avoid a patchwork of regulations.

by Thomas

ADHD Hyperfocus Engineers: A Double-Edged Sword

I recently came across an interesting topic: ADHD, often associated with inattentiveness, can paradoxically lead to intense hyperfocus in certain individuals. For engineers, this can be a double-edged sword. On one hand, hyperfocus can fuel extraordinary productivity and deep problem-solving. When engineers become engrossed in a challenging task, they can achieve remarkable feats.

However, hyperfocus can also lead to difficulties in managing time and prioritizing tasks. Engineers may become so absorbed in a project that they neglect other responsibilities or deadlines. Additionally, the intensity of hyperfocus can sometimes be emotionally draining, making it difficult to transition to other activities.

Engineers may find themselves working for hours without noticing the passage of time, completely immersed in their projects. This intense focus can lead to breakthroughs and innovative solutions and make significant contributions to their teams and projects.

by Thomas

How to use CodeT5 with LangChain

Here is how to create a CodeT5 wrapper for LangChain, which can be used to embed code generation, translation, or analysis tasks into your LangChain application.

Create a CodeT5 Custom LLM class for LangChain

from typing import Any

from langchain.llms.base import LLM
from langchain.prompts import PromptTemplate

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

class CodeT5LLM(LLM):
    model_name: str
    tokenizer: Any = None
    model: AutoModelForSeq2SeqLM = None
    
    def __init__(self, model_name):
        super().__init__(model_name=model_name)
        self.model_name = model_name
        self.tokenizer = AutoTokenizer.from_pretrained(model_name)
        self.model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

    def _call(self, prompt: str, stop=None) -> str:
        input_ids = self.tokenizer(prompt, return_tensors="pt").input_ids
        
        with torch.no_grad():
            generated_ids = self.model.generate(input_ids, max_length=128)
        
        return self.tokenizer.decode(generated_ids[0], skip_special_tokens=True)
    
    @property
    def _identifying_params(self):
        return {"model_name": self.model_name}

    def _llm_type(self):
        return "custom"

Usage of the CodeT5 LLM in your LangChain app

Then, you can use the CodeT5 LLM in your code like this:

codet5_llm = CodeT5LLM(model_name="Salesforce/codet5-base-multi-sum")

prompt = PromptTemplate(
    input_variables=["code"],
    template="{code}"
)

chain = prompt | codet5_llm
result = chain.invoke("def hello_world():\n    print('Hello, World!')")

print(result)

You may want to adjust the implementation based on your specific use case and the CodeT5 variant you’re using.

by Thomas

Explain like I am five: lvalue and rvalue in C++

Imagine you have a toy box.

  • Lvalue: Think of the toy box itself. It’s a place where you can keep your toys. You can put a toy in the box, take one out, or replace one toy with another. In C++, an lvalue is like this box — it’s a location in memory where a value is stored, and you can change what’s stored there.
    • Example: int my_number = 5;
    • Here, my_number is an lvalue because it’s like the toy box where the number 5 is stored. You can change my_numberlater, like putting a different toy in the box (e.g., my_number = 10;).
  • Rvalue: Now, think of a toy you’re holding in your hand. It’s a specific toy that you can use to play with, but you can’t change that toy into something else on the spot. In C++, an rvalue is like this toy— it’s a value that exists temporarily and can’t be directly modified.
    • Example: 5 + 3
    • The result of this (which is 8) is an rvalue. It’s like the toy you just created by combining parts (5 and 3), but you can’t change this 8 directly; you can’t say “8 = 10”. It’s just a value you use.

Key Points to Remember:

  • Lvalues can appear on the left side of an assignment (=). They represent things that can be modified or assigned to.
  • Rvalues typically appear on the right side of an assignment. They represent temporary values that can’t be assigned to directly.

Simplified Summary:

  • Lvalues are like toy boxes: You can store and change what’s inside.
  • Rvalues are like toys: You can use them, but you can’t change them directly.