“Concerns about privacy, reproducibility and cost are prompting scientists to use models on their computers rather than in the cloud.
When he wants to scratch an intellectual itch, Ian Gaudet does what people are increasingly doing on the Internet: he asks a chatbot. But when it comes to his research data, Gaudet, a neuroscientist at Florida Atlantic University in Boca Raton, prefers to stay offline. “I do not want Claude to start telling people my research results,” he explains.
AI co-scientists are revolutionizing how research is done
Cloud-based large language models (LLMs), such as those underlying the chatbots Claude (Anthropic), Gemini (Google) and ChatGPT (OpenAI), represent the cutting edge of artificial-intelligence performance. However, these systems might use what is typed into them for training purposes. For researchers such as Gaudet, state-of-the-art performance isn’t worth the risk to data protection, reproducibility or budgets.
Instead, they turn to a growing collection of tools, such as Ollama [1] and LM Studio [2], that enable researchers to download LLMs to their computer and use them offline.
These programs can help researchers to sort and query their own literature libraries to gather insights, for instance.
But two fresh barriers have replaced the programming and set-up issues that once kept researchers from using them: the hardware needed to run them has become much more expensive, and the tools’ output still requires scrutiny to assess its veracity.
Going local
Ask researchers why they want to run an AI model on their desktop rather than in the cloud, and the answer almost always starts with data. Eric Böhnisch-Volkmann, president of DEVONtechnologies in Coeur d’Alene, Idaho, hears it constantly from users of the firm’s document-management software DEVONthink, which integrates LLMs.
“Medical records is one thing, lawsuits is one other thing, research is another,” he says. “They just want to keep their hands on [their data] and not give it to someone else they don’t trust.”
Privacy concerns are well founded. A 2025 analysis found that the six AI developers studied — Amazon, Anthropic, Google, Meta, Microsoft and OpenAI — trained their models on users’ input data by default (J. King et al. Preprint at arXiv https://doi.org/g967b4; 2025).
AI for research: the ultimate guide to choosing the right tool
Ahmed Farooq, a computer scientist at Tampere University in Finland, built a local system called TAUCHI-GPT, so that papers, reports and study data protected under the European Union’s General Data Protection Regulation could be searched without leaving the university’s network.
Reproducibility is another benefit: cloud-based AI providers update their models without notice, so a query that works today might return different results next month. Casey Greene, a biomedical informatician at the University of Colorado Anschutz School of Medicine in Aurora, notes that if a local model converts his documents into a searchable index using the same settings each time, the results will always be identical. “If you want to use that for a scientific workflow, being able to know that is important,” he says.
Another benefit of local LLMs is economic. Jayashree Kalpathy-Cramer, who leads ophthalmology AI research at Anschutz, used Qwen (developed by Alibaba Cloud) and NVIDIA Nemotron through Ollama to index a decade’s worth of open-access publications on her machines. The corpus is public, so she could send it to a remote, commercial AI system.
But local inference is free, and that changes how she works.
Her system sends a daily briefing to her department’s chair highlighting new papers worth reading. The same query is run through three or four AI models, before another model reconciles the outputs, and every summarized finding is checked against the passage it came from. Because all of the field’s records sit in one searchable index, she can cluster them, map them and ask the LLM where any research gaps are.
The search for meaning
Preliminary findings of a survey by Wiley, a scientific publisher in Hoboken, New Jersey, indicate that 68% of some 2,400 researchers worldwide had used AI for literature searches or reviews (see go.nature.com/4rpuvwj). The tasks that were embraced fastest include reviewing large collections of studies, monitoring the latest publications and assisting with writing. Those are jobs that local models already do well, says Kalpathy-Cramer.
Greene built a local database of embeddings — numerical representations of each document’s overall meaning — to process the public abstracts of grants awarded to his university. When a federal-funding call for proposals in HIV research came through, he searched his database for potential collaborators. A keyword search for HIV highlighted a group researching the virus. But a semantic search of the same database, which looks at meaning rather than exact words, also came up with a researcher whose grants did not mention HIV in the title but who studied antiretroviral therapies, he says. He then asked that researcher about collaborating.
Dmitrijs Lvovs, a bioinformatician at the Institute for Genome Sciences at the University of Maryland School of Medicine in Baltimore, uses a computational pipeline to analyse spatial data from tumour tissue, producing hundreds of tables containing gene activity data, cell-type interaction information and records of treatment responses. To make sense of those data, he and his colleagues paste them into Microsoft Copilot, the cloud-based AI service that their university provides. The tool then searches the published literature and suggests what any patterns might mean. But Lvovs wants to bring that step in-house.
So, he started using a tool called llmize to feed the tables into Google Gemma 4 through Ollama, in a fully offline environment.
Because the model is not connected to the Internet, Gemma cannot search the literature in the way that Copilot can. But the tables represent people’s data, and Lvovs would rather accept a ‘weaker’ model than send such data sets to the cloud. Even if study participants have given their consent for data collection, “it feels wrong to be concentrating all that data in the hands of a few companies”, he says.
Getting started
In the world of LLMs, ‘weaker’ is a moving target, and the gap between local and cloud-based models is shrinking fast, Lvovs says. “The competition is now more about the harness, the actual program that uses the model as a back end, not the model itself.”
Forget ChatGPT: why researchers now run small AIs on their laptops
Most popular local AI platforms run on Mac, Windows and Linux operating systems.
Ollama, for instance, is free and can download and run models on all three.
In a video call, Lvovs typed “ollama run medgemma” into his computer’s command-line terminal to launch MedGemma, Google’s open model for medical text and imaging. He then popped a downloaded X-ray of a hand into the model; the response described the bones that the model ‘saw’. “That’s the only thing I need to do,” he says. “I don’t have to code.”
For people who avoid command lines, LM Studio by Element Labs in New York City provides a graphical interface for browsing, downloading and questioning models using a chat window, like a local ChatGPT. “I really wanted my own LLM,” explains creator Yagil Burowski, who built LM Studio so that anyone can try local models “without having to be super-duper experts”. The company’s Bionic agent, launched in July, can open, edit and work across files such as PDFs and spreadsheets on its own. Small models can run on the user’s machine. But Bionic’s largest models — those with hundreds of billions to trillions of parameters — require LM Studio’s cloud. (According to its launch announcement, the company neither retains uploaded data sets nor trains on them.)
Hardware requirements for local LLMs vary. Lvovs uses a Mac with 24 gigabytes (GB) of memory, and the smallest version of Gemma 4 can run on as little as 8 GB. But larger, more-reliable AI models need 18–20 GB. Sharon Machlis, a technology journalist in Boston, Massachusetts, who wrote a comprehensive how-to guide to local models for InfoWorld, found that AI tools work noticeably smoother on 32 GB than on 16 GB. “Sixteen is tough,” agrees Kalpathy-Cramer. “Sixty-four is much better.”
Testing limits
The boundary between what local models can handle well and what they struggle with is sharpest for work that requires judgement. Farooq and his colleagues found that, compared with cloud-based tools, a local AI system was significantly faster and produced higher satisfaction scores for editing and summarizing text (A. Farooq et al. Hum. Intell. Syst. Integr. 7, 53–85; 2025).
However, it performed worse than cloud-based tools for brainstorming and experimental design.
“If it’s a mission-critical result, you have to have a human expert looking at it,” he says. This applies to any LLM, local or cloud, he adds.
Any capable local model can summarize a document, says Machlis — although, she adds, she would not trust the results without checking them.
NatureTech hub
LLMs “can all hallucinate”, Kalpathy-Cramer says, which is why she checks every citation against the biomedical-science database PubMed. Tools for verifying citations include RefChecker, CheckIfExist, CiteAudit and sciwrite-lint. “For anything that I’m doing with a local model, I ground it,” she says. “I say, where in this PDF document did you find this information?”
And then there’s cost. The downloadable models with the highest capabilities need at least 96 GB of memory and the entry price of a computer powerful enough to use them “is probably about five grand”, says Christopher Thorpe, a computational biologist at the University of Oxford, UK. That is roughly the cost of NVIDIA’s DGX Spark computer (US$4,699), which Kalpathy-Cramer uses. Thorpe, who works with 32 GB of memory, finds that this is “not really enough to run a competitive model and have headroom to do other things”.
Prices show no sign of dropping, because AI companies’ demand for memory chips has sharply inflated consumer prices since mid-2025. Fortunately, people don’t need the most powerful models to organize and review the literature or to query PDFs, Lvovs says. His university’s computing cluster uses NVIDIA processors from 2017 that now sit mostly idle. “They’re not good any more for large tasks,” he says. “But for running a local model to query documents, they’re perfectly fine.”” [3]
1. Ollama is an open-source software platform for downloading, running, and managing large language models (LLMs) locally on your own computer or through hosted cloud models.
You can get started with the direct download software utility from Ollama or access installation guides and documentation on the Ollama GitHub repository.
Key Features
• Local & Cloud Execution: Runs open-weight models (such as Llama, Gemma, Qwen, DeepSeek, and Mistral) offline on macOS, Linux, and Windows, with optional hosted cloud models.
• Interfaces & API: Includes a command-line interface (CLI), [4] a desktop app, and a local REST API (defaulting to port 11434).
• Coding & Tool Integrations: Connects with coding agents and tools like Claude Code, Codex, and OpenCode, as well as SDKs for Python, JavaScript, Go, and more.
• Hardware Optimization: Uses quantization and supports Apple Metal, NVIDIA, AMD ROCm, and Vulkan to run models efficiently based on available memory.
Common CLI Commands
• ollama pull <model> — Download a model to your machine.
• ollama run <model> — Start an interactive session with a model.
• ollama list — View models installed locally.
2. LM Studio is a desktop application for downloading, managing, and running large language models (LLMs) locally on macOS, Windows, and Linux. It provides a graphical interface on top of backends like llama.cpp (GGUF) and MLX, allowing users to chat offline and run a local API server.
Core Capabilities
• Local Model Execution: Search and download models from sources like Hugging Face, configure context windows and parameters, and run prompts privately on your own hardware.
• Local API Server & CLI: Start a local server (lms server start) or manage models from the command line (lms ls, lms load) to connect local models with external tools and editors.
• LM Studio Bionic: A separate agentic app offering tool access to edit files, run commands, use an interactive canvas, and transcribe voice locally, while also supporting frontier cloud models with Zero Data Retention (ZDR).
Practical Considerations
• Privacy & Licensing: Once a model is downloaded, local inference stays on your device, though downloading models and using optional cloud features require a network connection. Check both LM Studio's terms and individual model licenses before commercial use.
• Backend Trade-offs: There is a consensus on r/LocalLLaMA that LM Studio's wrapped llama.cpp backend can lag behind upstream model releases, introduce occasional update regressions, and abstract away granular RAM/VRAM allocation metrics. As XDA Developers notes, Bionic is a distinct app from the classic model-chat and local-server desktop client.
3. Nature 658, 830-831 (2026) By David Brzostowicki
4. The best way to learn the command-line interface (CLI) is to start by practicing basic file navigation in a safe, sandboxed environment before moving on to real tasks on your own machine.
I would recommend starting with an interactive, browser-based playground so you can build muscle memory without worrying about accidentally deleting or moving important system files.
A. Open Your Terminal
Every operating system comes with a built-in CLI application:
• Mac: Open Spotlight (Cmd + Space), type Terminal, and press Enter.
• Linux: Press Ctrl + Alt + T.
• Windows: Open the Start menu and search for PowerShell. (Tip: If you are learning for web development or server management, consider installing Windows Subsystem for Linux (WSL) so you can practice standard Linux/Mac commands.)
B. Master the Core 10 Commands
Focus on learning how to look around and manage files first. Note that standard Mac/Linux (Bash/Zsh) commands differ slightly from traditional Windows Command Prompt (cmd), though modern PowerShell supports many of these aliases.
|
Action |
Mac / Linux (Bash/Zsh) |
Windows (CMD) |
|
See where you are (Print Working Directory) |
pwd |
cd |
|
List files & folders |
ls |
dir |
|
Change folders |
cd folder_name |
cd folder_name |
|
Go back one folder |
cd .. |
cd .. |
|
Create a new folder |
mkdir folder_name |
mkdir folder_name |
|
Create an empty file |
touch file.txt |
type nul > file.txt |
|
Copy a file |
cp source.txt dest.txt |
copy source.txt dest.txt |
|
Move or rename a file |
mv old.txt new.txt |
move old.txt new.txt |
|
Delete a file (Careful: no Trash bin!) |
rm file.txt |
del file.txt |
|
Get help on any command |
man command or command --help |
command /? |
C. Use Interactive Practice Tools
Instead of just reading documentation, use hands-on resources:
• Browser Sandboxes: Try the Command-line playground on sandbox.bio to run real commands directly in your web browser.
• Text-Based Games: Users on Reddit frequently recommend games like Bashcrawl, where you explore a dungeon by using cd, ls, and cat to move through rooms and read scrolls.
• Structured Courses: Follow the MDN Command Line Crash Course for a quick, developer-focused walkthrough, or read the free online book at LinuxCommand.org for deeper fundamentals.
D. Apply Essential Safety Habits
• Use Tab Auto-Complete: Press the Tab key while typing a file or folder name to let the terminal finish it for you—this saves time and prevents typos.
• Double-Check rm (Remove): Deleting files via the CLI bypasses your computer's Trash or Recycle Bin. Always check your current folder with pwd before deleting anything.
• Avoid sudo Until Needed: On Mac/Linux, sudo runs a command with administrator privileges. Never copy-paste a sudo command from the internet unless you understand what it does.
Komentarų nėra:
Rašyti komentarą