Stories
Slash Boxes
Comments

SoylentNews is people

You Can Now Run a GPT-3-Level AI Model on Your Laptop, Phone, and Raspberry Pi

posted by janrinok on Thursday March 16 2023, @06:24PM   Printer-friendly
from the welcome-future-dystopian-AI-overlords dept.

Freeman writes:

https://arstechnica.com/information-technology/2023/03/you-can-now-run-a-gpt-3-level-ai-model-on-your-laptop-phone-and-raspberry-pi/

Things are moving at lightning speed in AI Land. On Friday, a software developer named Georgi Gerganov created a tool called "llama.cpp" that can run Meta's new GPT-3-class AI large language model, LLaMA, locally on a Mac laptop. Soon thereafter, people worked out how to run LLaMA on Windows as well. Then someone showed it running on a Pixel 6 phone, and next came a Raspberry Pi (albeit running very slowly).

If this keeps up, we may be looking at a pocket-sized ChatGPT competitor before we know it.
[...]
For example, here's a list of notable LLaMA-related events based on a timeline Willison laid out in a Hacker News comment:

Related:
DuckDuckGo's New Wikipedia Summary Bot: "We Fully Expect It to Make Mistakes"
Robots Let ChatGPT Touch the Real World Thanks to Microsoft (Article has a bunch of other SoylentNews related links as well.)
Netflix Stirs Fears by Using AI-Assisted Background Art in Short Anime Film
Paper: Stable Diffusion "Memorizes" Some Images, Sparking Privacy Concerns
The EU's AI Act Could Have a Chilling Effect on Open Source Efforts, Experts Warn
Pixel Art Comes to Life: Fan Upgrades Classic MS-DOS Games With AI

Original Submission

 
This discussion was created by janrinok (52) for logged-in users only, but now has been archived. No new comments can be posted.
You Can Now Run a GPT-3-Level AI Model on Your Laptop, Phone, and Raspberry Pi | Log In/Create an Account | Top | 30 comments | Search Discussion
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

  • (Score: 1) by Runaway1956 on Friday March 17 2023, @12:36AM (4 children)

    by Runaway1956 (2926) Subscriber Badge on Friday March 17 2023, @12:36AM (#1296593) Journal

    Let me download and install an AI like ChatGomePhilTom. Do I get the source code? Well, personally, it doesn't really matter, because I can't read source well enough to decide if there is a back door or not. But, the point is, do I get the source code? I compile it myself? Or, is it an executable blob, and I'm just trusting the source? But, it's an AI, right? Who programmed it's ethics, exactly? Maybe calling home is the ethical thing to do?

    I'll pass. I don't want my chatbot informing the FBI and NSA of the locations of my nuclear arms caches. Or how much money I'm extorting from Hunter Biden, or . . . most of you should get the idea here.

  • (Score: 3, Informative) by Freeman on Friday March 17 2023, @02:22PM (3 children)

    by Freeman (732) on Friday March 17 2023, @02:22PM (#1296685) Journal

    https://github.com/tatsu-lab/stanford_alpaca [github.com] (Sounds like it's open-source with all the stuff you need to roll-your-own. With a possibility of them releasing the "secret sauce", if Meta says they can. Meta's "secret sauce" was already leaked and widely distributed.)

    Our initial release contains the data generation procedure, dataset, and training recipe. We intend to release the model weights if we are given permission to do so by the creators of LLaMA. For now, we have chosen to host a live demo to help readers better understand the capabilities and limits of Alpaca, as well as a way to help us better evaluate Alpaca's performance on a broader audience.

    Please read our release blog post https://crfm.stanford.edu/2023/03/13/alpaca.html [stanford.edu] for more details about the model, our discussion of the potential harm and limitations of Alpaca models, and our thought process for releasing a reproducible model.

    --
    Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"

    • (Score: 2, Informative) by guest reader on Friday March 17 2023, @06:42PM (2 children)

      by guest reader (26132) on Friday March 17 2023, @06:42PM (#1296728)

      Alpaca is "just" a fine-tuning of a LLaMA model.

      There is also another open source fine-tuning trainer ChatLLaMA [github.com].

      Both are based on pre-trained LLaMA models which means that you will still need to fill in Meta's form to obtain the LLaMA’s weights. Pre-trained LLaMA models have restrictive license: do not share, do not sue, nonpermanent, non-commercial use etc.

      The training of LLaMA is otherwise described in research paper LLaMA: Open and Efficient Foundation Language Models [facebook.com]. The training of 65B model took 21 days on 2048 A100 GPU cards. This article was a part of the LLaMA announcement [facebook.com].

      [...]Our training approach is similar to the methods described in previous work (Brown et al., 2020; Chowdhery et al., 2022), and is inspired by the Chinchilla scaling laws (Hoffmann et al., 2022). We train large transformers on a large quantity of textual data using a standard optimizer.

      [...]We preprocess five CommonCrawl dumps, ranging from 2017 to 2020, with the CCNet pipeline (Wenzek et al., 2020).

      [...]This process deduplicates the data at the line level, performs language identification with a fastText linear classifier to remove non-English pages and filters low quality content with an n-gram language mode

      [...]When training a 65B-parameter model, our code processes around 380 tokens/sec/GPU on 2048 A100 GPU with 80GB of RAM. This means that training over our dataset containing 1.4T tokens takes approximately 21 days.

      • (Score: 3, Informative) by coolgopher on Saturday March 18 2023, @01:08AM (1 child)

        by coolgopher (1157) on Saturday March 18 2023, @01:08AM (#1296787)

        Actually, you do not need to fill out their form; it's available via bittorrent as well [github.com].

        • (Score: 3, Interesting) by guest reader on Saturday March 18 2023, @07:21AM

          by guest reader (26132) on Saturday March 18 2023, @07:21AM (#1296833)

          Danger, Will Robinson.

          The BitTorrent link is just a pull request from a random guy. This pull request is not merged to the official LLaMA Meta Research repository.

          The official page [github.com] still contains the following instructions:

          In order to download the checkpoints and tokenizer, fill this google form.