{"schema":"libjg2-1", "vpath":"/git/", "avatar":"/git/avatar/", "alang":"", "gen_ut":1747289990, "reponame":"libclamma", "desc":"Modernized llama2.c inference engine", "owner": { "name": "Andy Green", "email": "andy@warmcat.com", "md5": "c50933ca2aa61e0fe2c43d46bb6b59cb" },"url":"https://warmcat.com/repo/libclamma", "f":3, "items": [ {"schema":"libjg2-1", "cid":"56b872836b7f0ec7e49c5546356a10bb", "commit": {"type":"commit", "time": 1690415929, "time_ofs": 0, "oid_tree": { "oid": "dad0ae393c16fa56329b5df7a840872be3563b45", "alias": []}, "oid":{ "oid": "7a4ca4a98b7dfc9d86f482b1d684e06a3330b6ec", "alias": []}, "msg": "add contributing section to readme, and also notable forks section", "sig_commit": { "git_time": { "time": 1690415929, "offset": 0 }, "name": "Andrej Karpathy", "email": "andrej.karpathy@gmail.com", "md5": "d06a9cdc46d537f09ccc4bd6b822dd78" }, "sig_author": { "git_time": { "time": 1690415929, "offset": 0 }, "name": "Andrej Karpathy", "email": "andrej.karpathy@gmail.com", "md5": "d06a9cdc46d537f09ccc4bd6b822dd78" }}, "body": "add contributing section to readme, and also notable forks section\n" , "diff": "diff --git a/README.md b/README.md\nindex 712786e..c0a6a92 100644\n--- a/README.md\n+++ b/README.md\n@@ -181,6 +181,28 @@ I trained the llama2.c storyteller models on a 4X A100 40GB box graciously provi\n \n Figured it's possible to reuse my existing discord channel (that I use for my [zero to hero youtube series](https://karpathy.ai/zero-to-hero.html)), see #llama2c channel on [discord](https://discord.gg/3zy8kqD9Cp), for any quick questions, related discussions, etc.\n \n+## contributing\n+\n+A few words on this repo and the kinds of PRs that are likely to be accepted. What is the goal of this repo? Basically I think there will be a lot of interest in training or finetuning custom micro-LLMs (think ~100M - ~1B params, but let's say up to ~10B params) across a large diversity of applications, and deploying them in edge-adjacent environments (think MCUs, phones, web browsers, laptops, etc.). I'd like this repo to be the simplest, smallest, most hackable repo to support this workflow, both training and inference. In particular, this repo is not a complex framework with a 1000 knobs controlling inscrutible code across a nested directory structure of hundreds of files. Instead, I expect most applications will wish to create a fork of this repo and hack it to their specific needs and deployment platforms.\n+\n+People who care about deployment efficiency above all else should look at [llama.cpp](https://github.com/ggerganov/llama.cpp). This repo still cares about efficiency, but not at the cost of simplicity, readability or portability. Basically, I expect that a lot of people come to this repo because the training code is 2 readable .py files and the inference code is 500 lines of C. So I'd like this to continue to be a kind of simplest \u0022reference implementation\u0022 that can be easily hacked in a separate fork into whatever downstream application people are excited about. It shouldn't be full-featured. It shouldn't take 100 different options or settings. It shouldn't be the most efficient. A few examples:\n+\n+- someone re-ordered two loops to improve data locality for a small efficieny win \u003d\u003e instant merge.\n+- someone added the one line \u0022pragma omp parallel for\u0022, which allows you to compile with OpenMP and dramatically speed up the code, or acts as just a comment if you don't compile it that way \u003d\u003e instant merge.\n+- bug fixes and touchups etc. \u003d\u003e happy to merge\n+\n+A few examples of PRs are that are not an excellent fit:\n+\n+- adding more than several #ifdefs all over the place in code. If they are localized / few, might be okay.\n+- adding a lot of code that is very specific to some specific platform (e.g. MCUs, or some special version of linux or processor). These may be a better fit for forks of the project, and I am very happy to maintain a list of these forks in section below.\n+- adding hundreds of lines of code to run.c that are only active in specific scenarios or platforms.\n+\n+If your candidate PRs have elements of these it doesn't mean they won't get merged, it just means they will make it into the gray territory. TLDR: I am eager to merge any mostly small, mostly localized, broadly applicable, clean changes that improve the efficiency and portability of the repo, while keep its hackability and readability. I appreciate all PRs seeking to help me improve the project, thank you! \u003c3.\n+\n+## notable forks\n+\n+- [llama2.rs](https://github.com/gaxler/llama2.rs) by @gaxler: a Rust port of this project\n+\n ## License\n \n MIT\n","s":{"c":1747289990,"u": 2547}} ],"g": 4013,"chitpc": 0,"ehitpc": 0,"indexed":0 , "ab": 0, "si": 0, "db":0, "di":0, "sat":0, "lfc": "0000"}