generative ai antimoats

Related

The Perfect Storm for Local AI Has Arrived by Rodney Waterman. The threshold at which owning hardware beats renting it is arriving faster than it used to. For sustained, high-utilization workloads, the time required to reach cost parity with cloud has compressed from around 15 months down to under four months….The quality story has changed too, drastically. Open-source models now handle 85 to 90 percent of enterprise AI use cases at levels indistinguishable from cloud APIs for most business applications.

How to roll your own local AI coding agents • The Register by Tobias Mann and Thomas Claburn. This isn’t the first time we’ve looked at local code assistants. Previously we explored using Continue’s VS Code extension for tasks such as code completion and generation. At the time, the models and software stack were quite immature, making them useful tools, but not necessarily good enough to compete with larger frontier models. Since then, model architectures and agent harnesses have improved dramatically.

I can now run a GPT-4 class model on my laptop The quality of models that are accessible on consumer hardware has improved dramatically in the past two years.

Hype, Sustainability, and the Price of the Bigger-is-Better Paradigm in AI The problem with the bigger-is-better approach is not simply that it is inaccessible. It also does not consistently produce the best models, and, after a given point, even shows diminishing returns. On many tasks, benchmark performance as a function of scale tends to saturate after a certain point (see Figure 2). In addition, there is as much variability in model performance between models within a similar size class as there is between models of different sizes.

Everything I’ve learned so far about running local LLMs I’m wary of vendor lock-in, having experienced the rug pulled out from under me by services shutting down, changing, or otherwise dropping my use case. I want the option to continue, even if it means changing providers. So for a couple of years I’d ignored LLMs. The “closed” models, accessibly only as a service, have the classic lock-in problem, including silent degradation. That changed when I learned I can run models close to the state-of-the-art on my own hardware—the exact opposite of vendor lock-in.

Big Tech Isn’t Prepared for A.I.’s Next Chapter by Bruce Schneier and Jim Waldo

Ask HN: Is it just me or GPT-4′s quality has significantly deteriorated lately? I saw the nerfing of GPT in real time: one day it was giving me great book summaries, the next one it said that it couldn’t do it due to copyright. I actually called it in a comment several months ago: copyright and other forms of control would make GPT dumb in the long run. We need an open source frontier less version.

Startups are using ChatGPT to meet soaring demand for chatbot therapy His experience mirrors that of other ChatGPT users on social media, who also reported that the chatbot was no longer engaging with them in automated therapy sessions.

It’s infuriatingly hard to understand how closed models train on their input These models aren’t yet anywhere near as capable as GPT-4, and claims that they compete with ChatGPT’s gpt-3.5-turbo mostly don’t hold up to deeper scrutiny. But… they’re pretty good—and they’re getting better at an impressive rate. And since you can run them on your own instances, they remove all possible concerns about what happens to the data that you pipe through them.

Will Open Source AI Shift Power from ‘Big Tech’? It Depends. The arc of open source development in software has taught us this much: the existence of open alternatives helps shape and constrain the power of market players, but doesn’t necessarily dictate a more diverse, competitive market structure.

gpt4all: an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue GPT4All is an ecosystem to train and deploy powerful and customized large language models that run locally on consumer grade CPUs….The goal is simple - be the best instruction tuned assistant-style language model that any person or enterprise can freely use, distribute and build on.

Why Purpose-Built Hardware is Critical to Advancing Generative AI | Ars Technica (Amazon hosts all the other random open-source server software as SaaS, why not AI too?)

Andreessen Horowitz Funds ‘Uncensored’ AI That Will Tell You How to Kill Yourself Thankfully, the open source ecosystem is starting to develop, and we are now seeing open source models that rival closed-source alternatives.

Shares in China’s iFlyTek tumble after reports AI-powered device criticised Mao | Reuters

Model alignment protects against accidental harms, not intentional ones Even if there is draconian regulation to prohibit the release of new open models, and global cooperation to enforce such regulations, the open models that have already been released are more than sufficient for harmful applications like disinformation and scams.

Phi-2: The surprising power of small language models - Microsoft Research We are now releasing Phi-2, a 2.7 billion-parameter language model that demonstrates outstanding reasoning and language understanding capabilities, showcasing state-of-the-art performance among base language models with less than 13 billion parameters. On complex benchmarks Phi-2 matches or outperforms models up to 25x larger, thanks to new innovations in model scaling and training data curation.

Founder of Neo-Nazi Group the Base Instructs Followers to Use ‘Uncensored’ AI The news shows how AI models and AI internet platforms with less restrictions compared to market leaders like OpenAI are making it increasingly easier for anyone to generate any kind of content, and that leaders of extremist movements have considered how it might be useful for them.

Microsoft Deleted Its LLM Because It Didn’t Get a Safety Test, But Now It’s Everywhere Last week, Microsoft researchers released WizardLM 2, which it claimed is one of the most powerful open source large language models to date. Then it deleted the model from the internet a few hours later because, as The Information reported, it “accidentally missed” required “toxicity testing” before it was released.

Why I Use Open Weights LLMs Locally and 50+ Open-Source Options for Running LLMs Locally by Vince Lam You’ve likely noticed that proprietary models are highly censored, limited, and biased due to alignment — to reduce harmful and toxic responses….Now this is not an inherently bad, but sometimes the LLM may refuse to fulfil a task even if it seems reasonable, or it may simply not infer on certain topics. and By using mostly free models and occasionally switching to GPT-4, my monthly expenses dropped from 20 USD to 0.50 USD.

Forget ChatGPT: why researchers now run small AIs on their laptops First, organizations are making open weights versions of LLMs, in which the weights and biases used to train a model are publicly available, so that users can download and run them locally, if they have the computing power. Second, technology firms are making scaled-down versions that can be run on consumer hardware — and that rival the performance of older, larger models. Researchers might use such tools to save money, protect the confidentiality of patients or corporations, or ensure reproducibility.

Chinese AI startup DeepSeek unveils open-source model to rival OpenAI o1 – Computerworld With its MIT license and ultra-low costs, DeepSeek could be an appealing and cost-effective option for enterprise adoption. Chinese AI developer DeepSeek has unveiled an open-source version of its reasoning model, DeepSeek-R1, featuring 671 billion parameters and claiming performance superior to OpenAI’s o1 on key benchmarks.

Copyright reform is necessary for national security - Anna’s Blog Virtually all major companies building LLMs contacted us to train on our data. Most (but not all!) US-based companies reconsidered once they realized the illegal nature of our work. By contrast, Chinese firms have enthusiastically embraced our collection, apparently untroubled by its legality….We have given high-speed access to about 30 companies. Most of them are LLM companies, and some are data brokers, who will resell our collection. Most are Chinese, though we’ve also worked with companies from the US, Europe, Russia, South Korea, and Japan.

Ollama - NSHipster Ollama is the easiest way to run large language models on your Mac. You can think of it as “Docker for LLMs” - a way to pull, run, and manage AI models as easily as containers.

DeepSeek goes beyond “open weights” AI with plans for source code release The open source release could also help provide wider and easier access to DeepSeek even as its mobile app is facing international restrictions over privacy concerns.

The DeepSeek Effect: Some Large Cloud Customers Slow Down AI Spending as Prices Drop — The Information Thomson Reuters released conversational AI in its tax and legal software products in late 2023 to help customers draft legal documents and answer questions about tax laws. But its overall spending on AI from cloud providers like Microsoft and Google has been roughly flat since then, thanks to the falling price of the models powering the features, which include those made by Meta Platforms, Google and DeepSeek, according to Chief Technology Officer Joel Hron. If anything, the gap has shrunk between the open-source models and what OpenAI or Anthropic are able to do, Hron said. And with that, there’s been a precipitous drop in cost.

More of Silicon Valley is building on free Chinese AI by Jasmine Cui and Jared Perlo. NBC News spoke to over 15 AI startup founders, machine-learning engineers, industry experts and investors, who said that while models from American companies continue to set the pace of progress at the frontier of AI capabilities, many Chinese systems are cheaper to access, more customizable and have become sufficiently capable for many uses over the past year.

Bonus links

Google promised to delete sensitive data. It logged my abortion clinic visit

It’s Time to Acknowledge Big Tech Was Always at Odds with Journalism

Washington’s My Health My Data Act: Welcome to BIPA 2.0 (when the surveillance advertisers are freaking out this much about your law, it’s probably a good law imho)

Will A.I. Become the New McKinsey? | The New Yorker

Tech job opportunities remain strong despite layoffs — here’s why

Facebook, Youtube and TikTok content moderators in Kenya form labour union

OpenAI’s hunger for data is coming back to bite it

Leaders, your mass layoffs may have an unintended side effect that hurts the company’s bottom line

ChatGPT creates mostly insecure code, but won’t tell you unless you ask

AI Drake Just Set An Impossible Legal Trap For Google, by Nilay Patel, The Verge