NotesTo me, all signs point towards software engineering changing radically as a profession to be much more oriented around the what and why of software, and much less around the how. This will cause disruption at a massive scale in the long run. But in the short run, it's just a lot of fun to play with these tools and see what they can do.Unfurl
NotesIt is surprisingly straightforward to increase the VRAM of your Mac (Apple Silicone M1/M2/M3 chips) computer and use it to load large language models. Hereās the rundown of my experiments. ... I found a way to bypass this limitation. To allocate more of your Macās system RAM to VRAM ā in this case, up to 28 GB ā the following command can be used in the terminal window:
sudo sysctl iogpu.wired_limit_mb=27536FeedEmbedUnfurl
Notesmy attempt to build such a capable AI computer without spending too much. I ended up with a workstation with 48GB of VRAM that cost me around 1700 eurosFeedUnfurl
NotesYou wrote an interesting comment about getting your work into the LLM training corpus: "there has never been a more vital hinge-y time to write."
Do you mean that in the sense that you will be this drop in the bucket thatās steering the Shoggoth one way or the other? Or do you mean it in the sense of making sure your values and persona persist somewhere in latent space?FeedEmbedUnfurl
NotesPersonally, I feel like I get a lot of value from AI. I think many of the people who donāt feel this way are āholding it wrongā: i.e. theyāre not using language models in the most helpful ways. In this post, Iām going to list a bunch of ways I regularly use AI in my day-to-day as a staff engineer.FeedUnfurl
NotesUndoubtedly, the sloppification of the internet will likely get worse over the next few years. And as such, the returns to curating quality sources of content will only increase. My advice? Use an RSS feed reader, read Twitter lists instead of feeds, and find spaces where real discussion still happens (e.g. LessWrong and Lobsters still both seem slop-free).Unfurl
NotesAt the same time, there should be some humility about the fact that earlier iterations of the chip ban seem to have directly led to DeepSeekās innovations. Those innovations, moreover, would extend to not just smuggled Nvidia chips or nerfed ones like the H800, but to Huaweiās Ascend chips as well. Indeed, you can very much make the case that the primary outcome of the chip ban is todayās crash in Nvidiaās stock price.Unfurl
NotesPeople with less knowledge about AI are actually more open to using the technology. We call this difference in adoption propensity the ālower literacy-higher receptivityā link.Unfurl
NotesI feel like half of my social media feed is composed of AI grifters saying software developers are not going to make it. Combine that sentiment with some economic headwinds and it's easy to feel like we're all screwed. I think that's bullshit. The best days of our industry lie ahead.Unfurl
NotesJust as Midas discovered that turning everything to gold wasn't always helpful, we'll see that blindly applying cosine similarity to vectors can lead us astray. While embeddings do capture similarities, they often reflect the wrong kind - matching questions to questions rather than questions to answers, or getting distracted by superficial patterns like writing style and typos rather than meaning. This post shows you how to be more intentional about similarity and get better results.
NotesThis document is a summary of my personal experiences using generative models while programming over the past year. It has not been a passive process. I have intentionally sought ways to use LLMs while programming to learn about them. The result has been that I now regularly use LLMs while working and I consider their benefits net-positive on my productivity. (My attempts to go back to programming without them are unpleasant.)FeedUnfurl
NotesWelcome to OpenHands (formerly OpenDevin), a platform for software development agents powered by AI.
OpenHands agents can do anything a human developer can: modify code, run commands, browse the web, call APIs, and yesāeven copy code snippets from StackOverflow.Unfurl
NotesPaddleOCR aims to create multilingual, awesome, leading, and practical OCR tools that help users train better models and apply them into practice.Unfurl
NotesConventional search methods rely on keyword matching, where the system locates exact words or phrases from the query within documents. This technique can be enhanced to better capture the context and intent behind the user's query, leading to more relevant and precise search results. Semantic search focuses on understanding the meaning and intent behind the query. Combining semantic search with filtersāor additional parameters to narrow the results based on specific attributesāfurther improves accuracy.
In this article, we explore semantic search with filters and demonstrate how you can implement it using pgvector and JavaScript.FeedUnfurl
NotesAI isn't making our software dramatically better because software quality was (perhaps) never primarily limited by coding speed. The hard parts of software development ā understanding requirements, designing maintainable systems, handling edge cases, ensuring security and performance ā still require human judgment.
What AI does do is let us iterate and experiment faster, potentially leading to better solutions through more rapid exploration. But only if we maintain our engineering discipline and use AI as a tool, not a replacement for good software practices. Remember: The goal isn't to write more code faster. It's to build better software. Used wisely, AI can help us do that. But it's still up to us to know what "better" means and how to achieve it.FeedUnfurl
NotesNew, more powerful chips require entirely new methods to rack-mount, operate and cool them, and all of these parts must operate in sync, as overheating GPUs will die. While these units are big, some of their internal components are microscopic in size, and unless properly cooled, their circuits will start to crumble when roasted by a guy typing "Garfield with Gun" into ChatGPT.FeedUnfurl
NotesI hope that you now have a sense for word embeddings and the word2vec algorithm. I also hope that now when you read a paper mentioning āskip gram with negative samplingā (SGNS) (like the recommendation system papers at the top), that you have a better sense for these concepts.FeedUnfurl
NotesSo the problem Bluesky is dealing with is not so much a problem with Bluesky itself or its architecture, but one thatās inherent to the web itself and the nature of building these training datasets based on publicly-available data. Van Strienās original act clearly showed the difference in culture between AI and open social web communities: on the former itās commonplace to grab data if it can be read publicly (or even sometimes if itās not), regardless of licensing or author consent, while on open social networks consent and authorsā rights are central community norms.FeedUnfurl
NotesSo, in summary: maybe people shy away from copilots because theyāre tired of complexity, theyāre tired of accelerating productivity without improving hours, theyāre afraid of forgetting rote skills and basic knowledge, and they want to feel like writers, not managers.
Maybe some or none of these things are true - theyāre emotional responses and gut feelings based on predictions - but they matter nonetheless.FeedUnfurl
NotesA newsletter from author Steven Johnson exploring where good ideas come fromāand how to keep them from turning against us. Click to read Adjacent Possible, by Steven Johnson, a Substack publication with tens of thousands of subscribers.FeedUnfurl
NotesThe current state-of-the-art Gemini model can fit roughly 1.5 million words in its context. Thatās enough for me to upload the full text of all fourteen of my books, plus every article, blog post, or interview Iāve ever publishedāand the entirety of my collection of research notes that Iāve compiled over the years. The Gemini team has announced plans for a model that could hold more than 7 million words in its short-term memory. Thatās enough to fit everything Iāve ever written, plus the hundred books and articles that most profoundly shaped my thinking over the years. An advanced model capable of holding in focus all that information would have a profound familiarity with all the words and ideas that have shaped my personal mindset. Certainly its ability to provide accurate and properly-cited answers to questions about my worldview (or my intellectual worldview, at least) would exceed that of any other human. In some ways it would exceed my own knowledge, thanks to its ability to instantly recall facts from books I read twenty years ago, or make new associations between ideas that I have long since forgotten. It would lack any information about my personal or emotional historyāthough I suppose if I had maintained a private journal over the past decades it would be able to approximate that part of my mindset as well. But as reconstruction of my intellectual grounding, it would be unrivaled. If that is not considered material progress in AI, there is something wrong with our metrics.Unfurl
NotesIn this guide we'll discuss:
Where and when fine-tuning can be useful.
Alternative approaches to extending the capabilities and behavior of pre-trained models.
The importance of data preparation.
How to fine-tune Mistral 7B using your own custom dataset with Axolotl.
The many hyperparameters and their effect on training.
Additional resources to help you fine-tune your models faster and more efficiently.FeedUnfurl
NotesWeb Applets are small, secure pieces of web code (bundles of HTML, JavaScript, and CSS) that can run anywhere, allowing a model to take actions within software much like a human would and then generate interfaces appropriate for the userās intent. For example, a developer could write an applet that enables a model to respond to a query about local coffee shops by conducting internet searches and then displaying the results on an in-line map. And because the model can read the internal state of each applet, it can then conduct follow-up actions to complete a userās request (for example, updating the map to display only coffee shops that will be open tomorrow afternoon). Anyone can build Web Applets and host them on the Web, and any client can potentially support them.Unfurl
NotesIf, as Marx argued, capital is dead labor, then the products of large language models might best be understood as dead speech. Just as factory workers produce, with their āliving labor,ā machines and other forms of physical capital that are then used, as ādead labor,ā to produce more physical commodities, so human expressions of thought and creativityāāliving speechā in the forms of writing, art, photography, and musicābecome raw materials used to produce ādead speechā in those same forms. LLMs, to continue with Marxās horror-story metaphor, feed āvampire-likeā on human culture. Without our words and pictures and songs, they would cease to function. They would become as silent as a corpse in a casket.FeedUnfurl
NotesOver the past month Iāve been exploring the rapidly evolving world of Large Language Models (LLM). Itās now accessible enough to run a LLM on a Raspberry Pi smarter than the original ChatGPT (November 2022). A modest desktop or laptop supports even smarter AI. Itās also private, offline, unlimited, and registration-free. The technology is improving at breakneck speed, and information is outdated in a matter of months. This article snapshots my practical, hands-on knowledge and experiences ā information I wish I had when starting. Keep in mind that Iām a LLM layman, I have no novel insights to share, and itās likely Iāve misunderstood certain aspects. In a year this article will mostly be a historical footnote, which is simultaneously exciting and scary.FeedUnfurl
NotesIāve been having quite a bit of fun with the fairly recent LivePortrait model, generating deepfakes of my friends for some cheap laughs.FeedUnfurl
Notes Large language models (LLMs) have shown remarkable proficiency in generating text, benefiting from extensive training on vast textual corpora. However, LLMs may also acquire unwanted behaviors from the diverse and sensitive nature of their training data, which can include copyrighted and private content. Machine unlearning has been introduced as a viable solution to remove the influence of such problematic content without the need for costly and time-consuming retraining. This process aims to erase specific knowledge from LLMs while preserving as much model utility as possible. Despite the effectiveness of current unlearning methods, little attention has been given to whether existing unlearning methods for LLMs truly achieve forgetting or merely hide the knowledge, which current unlearning benchmarks fail to detect. This paper reveals that applying quantization to models that have undergone unlearning can restore the "forgotten" information. To thoroughly evaluate this phenomenon, we conduct comprehensive experiments using various quantization techniques across multiple precision levels. We find that for unlearning methods with utility constraints, the unlearned model retains an average of 21\% of the intended forgotten knowledge in full precision, which significantly increases to 83\% after 4-bit quantization. Based on our empirical findings, we provide a theoretical explanation for the observed phenomenon and propose a quantization-robust unlearning strategy to mitigate this intricate issue... Unfurl
NotesIt is nightmarish to me to read reports of how reliant on ChatGPT students have become, even outsourcing to the machines the ideally very personal assignment "briefly introduce yourself and say what you're hoping to get out of this class." It is depressing to me to read defenses of those students, particularly this one that compares an AI-written essay to using a washing machine in that it reduces the time required for the labor. This makes sense only if the purpose of a student writing an essay is "to have written an essay," which it is not. The teacher did not assign it as busywork. The purpose of an essay is to learn and practice communication skills, critical thinking, organization of one's own thoughts. These are useful skills to develop, even (especially!) if you do not go into a writing career. Unfurl
NotesOasis takes in user keyboard input and generates real-time gameplay, including physics, game rules, and graphics. You can move around, jump, pick up items, break blocks, and more. There is no game engine; just a foundation model.Unfurl
NotesA more effective abstraction is conceptualizing vector embeddings not as independent tables or data types but as a specialized index on the embedded data. This is not to say that vector embeddings are literally indexes in the traditional sense, like those in PostgreSQL or MySQL, which retrieve entire data rows from indexed tables. Instead, vector embeddings function as an indexing mechanism that retrieves the most relevant parts of the data based on its embeddings.FeedUnfurl
NotesI released LLM 0.17 last night, the latest version of my combined CLI tool and Python library for interacting with hundreds of different Large Language Models such as GPT-4o, Llama, Claude and Gemini.FeedUnfurl
Notes Embeddings aren't exactly new, but they have become much more widely accessible in the last couple years. What embeddings offer to technical writers is the ability to discover connections between texts at previously impossible scales.FeedUnfurl
NotesGenerative AI is like if capitalism reinvented the fae. Itāll trick you into accepting its agreement and then itāll steal your face and start speaking with your voice.EmbedUnfurl
NotesIlya Sutskever shared a list of 30 papers with John Carmack and said, āIf you really learn all of these, youāll know 90% of what matters todayāFeedUnfurl
NotesThis is how weāre headed for another AI winter, just as we saw with the fall of data science, crypto, and the modern data stack. And thatās actually a good thing. The promoters will hop onto the next trendy buzzword, while the real producers will keep moving forward, building a more capable future for AI.FeedUnfurl
Notessqlite-lembed is a SQLite extension for generating text embeddings, meant to work alongside sqlite-vec. With a single embeddings model file provided in the .gguf format, you can generate embeddings using regular SQL functions, and store them directly inside your SQLite database. No extra server, process, or configuration needed!Unfurl
NotesThe most common use-case for vectors in databases is for K-nearest-neighbors (KNN) queries. You'll have a table of vectors, and you'll want to find the K closestUnfurl
NotesYou can use SQLite's builtin full-text search (FTS5) extension and semantic search with sqlite-vec to create "hybrid search" in your applications. You can combine results using different methods like keyword-first, re-ranking by "semantics", and reciprocal rank fusion. Best of all, since it's all in SQLite, experiments and prototypes are cheap and easy, no 3rd party services required!Unfurl
NotesThis tutorial covers the origins and uses of the BART model for text summarization tasks, and concludes with a brief demo for using BART with Paperspace Notebooks.Unfurl
NotesWe have entered an era of LLM democratization. By showing that smaller models can be highly effective, enabling easy experimentation, diversifying control, and providing incentives that are not profit motivated, open-source initiatives are moving us into a more dynamic and inclusive AI landscape. This doesnāt mean that some of these models wonāt be biased, or wrong, or used to generate disinformation or abuse. But it does mean that controlling this technology is going to take an entirely different approach than regulating the large players.FeedEmbedUnfurl
NotesBut in calling these programs āartificial intelligenceā we grant them a claim to authorship that is simply untrue. Each of those tokens used by programs like ChatGPTāthe ālanguageā in their ālarge language modelāārepresents a tiny, tiny piece of material that someone else created. And those authors are not credited for it, paid for it or asked permission for its use. In a sense, these machine-learning bots are actually the most advanced form of a chop shop: They steal material from creators (that is, they use it without permission), cut that material into parts so small that no one can trace them and then repurpose them to form new products.Unfurl
NotesItās possible that, in the future, we will build an A.I. that is capable of writing good prose based on nothing but its own experience of the world. The day we achieve that will be momentous indeedābut that day lies far beyond our prediction horizon. In the meantime, itās reasonable to ask, What use is there in having something that rephrases the Web?FeedUnfurl
NotesSooner or later, every single conversation I have will be recorded and transcribed and Iāll be able to look back at it later ā details from a phone call with the bank, in the hardware store asking a question, someone mentions a book at the pub, an idea in a workshop. Ignoring the societal consequences for a sec lol ahemā¦ how should the app to manage all that chatter work?FeedUnfurl
How to Increase the VRAM of Your Mac with Apple Silicone for LLMs? | Hardware Corner
ewintr.nl - building a personal, private ai computer on a budget
Gwern Branwen - How an Anonymous Researcher Predicted AI's Trajectory
AI Slop, Suspicion, and Writing Back | Ben Congdon
DeepSeek FAQ ā Stratechery by Ben Thompson
Knowing less about AI makes people more open to having it in their lives ā new research
Ignore the Grifters - AI Isn't Going to Kill the Software Industry ā Dustin Ewers
Home - PaddleOCR Documentation
Implementing Filtered Semantic Search Using Pgvector and JavaScript
The 70% problem: Hard truths about AI-assisted coding
Godot Isn't Making it
The Illustrated Word2vec ā Jay Alammar ā Visualizing machine learning one concept at a time.
Bluesky, AI, and the battle for consent on the open web
Between the Booms: AI in Winter ā Communications of the ACM
Adjacent Possible | Steven Johnson | Substack
You Exist In The Long Context
An introduction to fine-tuning LLMs at home with Axolotl ā¢ The Register
Sponsoring the Web Applets project, an open approach to AI-empowered web apps - Mozilla Innovations
Dead Labor, Dead Speech - by Nicholas Carr
Perceptually lossless (talking head) video compression at 22kbit/s | Martin Lumiste
[2410.16454] Does your LLM truly unlearn? An embarrassingly simple approach to recover unlearned knowledge
Whatever AI Looks Like, It's Not | Defector
Oasis
Vector Databases Are the Wrong Abstraction
You can now run prompts against images, audio and video in your terminal using LLM
The A.I. Bubble is Bursting with Ed Zitron - YouTube
Embeddings are underrated
Aman's AI Journal ā¢ Primers ā¢ Ilya Sutskever's Top 30
AI Winter Is Coming
Introducing sqlite-lembed: A SQLite extension for generating text embeddings locally | Alex Garcia's Blog
KNN queries | sqlite-vec
Hybrid full-text search and vector search with SQLite | Alex Garcia's Blog
Using Llamafiles for Embeddings in Local RAG Applications
BART Model for Text Summarization
Overview ā Ray 2.34.0
Open-Source LLMs - Schneier on Security
ChatGPT is not āartificial intelligence.ā Itās theft. | America Magazine
ChatGPT Is a Blurry JPEG of the Web | The New Yorker
Transcribing all our conversations 24/7 will be weird and also useful maybe (Interconnected)