A New Scientific Operating System

Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown. - Vannevar Bush

In the past, scientists like Newton would withhold sharing findings and instead only share anagrams so they could later prove priority on discovery. It wasn't until the advent of journals that science became more open with more sharing across scientists in various disciplines (see The Royal Society and others).

Today, we take much of this history of science for granted. Breakthroughs like CRISPR, the Human Genome Project, and CAR-T cell therapy demonstrate science’s impact on our lives. But behind the scenes, scientists operate within institutional dynamics that stifle their creativity and ability to pursue transformative science.

Efforts like Open Science have chipped away at these problems, but institutional inertia has limited progress. By pulling together ideas from Open Science and other recent proposals, I argue that two recent shifts -- 1) political upheaval in U.S. federal funding and 2) the rise of generative AI -- create a rare opening for new, enduring models of research funding and infrastructure.

I further explore how we got to this point, and what a potential, new Scientific Operating System (SciOS) -- meaning the infrastructure, tools, norms, and institutions that support the full lifecycle of scientific inquiry -- for basic research might entail.

Existing Framework for Scientific Progress

Nurturing systems to promote scientific inquiry and progress -- with respect to research direction, advancement of knowledge, development of technologies in public interest, and the nurture of scientific talent -- is a key concern of any society.

After World War 2, Vannevar Bush wrote his famous memo, Science, The Endless Frontier to outline the importance of a framework for relationships between the federal government and universities to maintain global relevance for the United States.

This led to the creation of a framework where the federal government funded basic research done at universities (even without immediate, short-term impact) and ensured we had a pipeline for domestic talent.

Shortly after, we had the Bayh-Dole Act, which intended to ensure that tax-payer funded research would not only advance knowledge, but be translated into technologies, products, or services that benefitted the public.

Universities not only became key pipelines of scientific talent and centers of basic research, but evolved into engines of innovation through Technology Transfer Offices (TTO).

As a result, you can imagine universities having 2 core functions:

1. Serving as the bedrock for science -- i.e. both the development of new scientists, and conducting basic research without short term ROI, which was largely cemented in the years after Vannevar Bush's memo.
2. Ensuring research with potential can be translated / commercialized by working with governmental agencies (think NIH) or industry.

Over time, they continued to bundle more and more functions into their overall purpose in society, while pulling other groups into the orbit of this budding scientific system. Most notable were scholarly societies and publishers, which started well before as a way to disseminate scientific knowledge, but increasingly cemented themselves with universities to influence how science is done, and how new knowledge is, and is not, shared.

"Science" Today

Though journals played an important role in opening up science, the publishers behind them needed to maintain their relevance even in the internet age.

They are, after all, businesses, with a mandate to not just exist, but grow. One way to create staying power as a publisher is to:

• Monetize access to read, or publish content, which you can defend through copyright laws
• Ensure that people have to publish with you by:
  • Creating prestige by defining different levels of importance for journals
  • Building a network of reviewers to review content ready for publication to ensure it's "held to a necessary standard"
  • Ensuring that citations -- the primary mechanism to build on scientific knowledge -- becomes a currency for one's career through impact, rankings, and more, by working with indices and agencies that track these measures
  • Creating secondary mechanisms of career advancement -- through professional networks, and more, mediated by your society.

This is not an evil scheme by publishers tricking everyone else, but an optimization by all players around the perverse incentives they have.

Consider the perspective of a university, which gets significant revenue through taxpayer dollars via indirect costs on grants: how do we recruit and retain faculty securing those grants?

• Tie faculty tenure to publications, impact, and prestige (with the help of publishers, ranking agencies, and more)
• Work with citation indices and publishers to improve the "brand value" of the university to attract new faculty that might bring more grant revenue

And so, journals and publishers, while serving an important, core need, have gradually morphed the system with universities and federal funding into something with a number of issues:

• The creation of silos of information that are not interoperable between research groups
• A disincentive to publish important findings. Why share something as soon as you have it, if you can mine it for a little more to get ahead of someone else?
• A disincentive to publish negative results
• An inefficient peer-review system built on underpaid (or unpaid) labor, which simply cannot scale with the exponential rate of increase in new publications (At a recent conference, someone proposed that publishers pay for coffee for reviewers!)
• An incentive to falsify results -- in egregious but also in subtle ways -- to get more funding.
• Being published is treated like getting a stamp of approval.
• A narrative driven format for scientific findings, instead of a focus on the facts and "knowledge units"

Are there some disciplines in science where these problems are not as big?

I think so, especially in technical fields which have been influenced by Open Source software, which made it easier to embrace Open Science methodology (physics, computer science, ML/AI).

Yet other fields with high impact on society -- life sciences, etc. remain plagued by these issues. Attempts to address these from within are met with significant inertia, making any meaningful change to adapt the system to the digital age virtually impossible.

A Shifting Federal Landscape

Recent political events that threaten federal research funding and weaponize the federal government against universities are arguably a black swan event. As the federal framework used to fund science is being dismantled, universities are looking for alternative sources of revenue while facing an unbunding of different functions they took on in the last few decades.

Time will tell exactly how this plays out, but these changes are already opening the door for an unbundling of the university TTO function I referenced earlier (core function #2 from earlier). Ecosystem developers like Portal Innovations as well as venture / industry partners are finding new ways to work with universities to help them diversify their sources of revenue through the IP they have.

What I find more interesting is that the shakiness of the federal funding environment makes universities and researchers increasingly more open to exploring new sources of funding for basic research (not just commercialization). That means an opening to work past the institutional inertia created by universities, publishers, etc. which preserves incentives that work against great science (this inertia makes it difficult to improve core function #1 above).

Don't get me wrong. All of this alone is insufficient to topple over an old system and replace it with something new. But that's also besides the point.

What we have, in this moment, is a very, very useful wedge to get scientists and universities open minded about new opportunities to fund their interests, which enables groups like Astera Institute or Convergent Research to experiment with new ways of funding science, on their own terms, and opens the door to rethink how federal funding works (check out the X-Labs proposal or ideas to experiment with NIH funding).

How do we ensure that these new models -- from a mix of funding individuals, teams, or more -- if promising, actually stay and work in service of great science? And if they do stay, what foundation needs to be in place to start moving the needle on these old incentive structures?

The Generative AI Moment

Like other industries, many problems within science are structural. They aren't technology problems to be solved through a new app. But, leveraging technology to support a new, modern infrastructure for more open, collaborative science can create the structural shift the system needs and give these new models the necessary power to stay.

While uncertainty at the federal level is creating one set of opportunities around new funding models and infrastructure for basic science, recent advances in large language models unlock our ability to new approaches real staying power.

Many of the mechanisms we've built were predicated on data and information being worked on at what I call people scale: the natural limits at which a group of people can reasonably organize and do things (think analysts, peer review, etc.).

What does a new Scientific Operating System look like that preserves the basic tenets of scientific inquiry, without compromising the benefits of science for humanity?

And how can we ensure that it continues to celebrate the people involved in doing great science, without putting the individual above the institution?

A well-designed Scientific Operating System — meaning the infrastructure, tools, norms, and institutions that support the full lifecycle of scientific inquiry — should optimize for truth-seeking, openness, collaboration, and impact.

In essence, it should let good science happen faster, more reliably, and more inclusively.

Stated differently, such a system should:

• Continue to fund exploratory research, even without a clear short or medium term ROI, see Astera Institute for a good example
• Be scientific, experimentative, and introspective about its own performance
• Encourage the publication of negative results, which over time will speed up the rate of innovation (white space problem)
• Support the publication of smaller units of results (knowledge units), not just fully formed narratives
• Allow for continuous iteration -- meaning publishing intermediate results
• Make all findings accessible, discoverable, and machine-readable
• Make things like replication checks, data sharing, etc. native to the system
• Create norms that reward and incentivize openness instead of secrecy
• Ensure scientists can build their prestige in an interpretable way through more granular attribution for their contributions, instead of it being transferred in aggregate by journal impact and authorship position

Though the technology and infrastructure was always possible for some of these (and in fact, is slowly being built -- see DataCite, Crossref, OpenAlex, preprint servers), it feels much more realistic to scale these systems in an agentic world, especially if we explore creating a more granular unit of science for more rapid / iterative sharing of results.

Large language models are getting us to a point where we can:

• Address issues of information overload through agentic systems that can contextually search through and evaluate the relevance of significantly more articles than an individual human (see Undermind and FutureHouse for deep, adaptive scientific search, or scite.ai for innovations in citation indices)
• Have new ways to formulate hypotheses and analyze data (consider PlutoBio, PotatoAI, and Elicit)
• Make it easier to harmonize different data sources (see DevanoAI)
• Run faster experiments that we can scale through machine-readable, interoperable data
• Streamline how we communicate findings and build on our existing knowledge

When new models of funding science are combined with new infrastructure and new technology, there's an opportunity to transform how basic science is done.

A Once in a Century Rewrite

Scientists are hackers and painters. They are creative, intelligent, logical. They want to express themselves and solve interesting, potentially transformative problems. The incumbent system can work against this core desire (to varying degrees in different disciplines).

A new Scientific Operating System isn't just a new toolset, or just a new methodology. It's a set of interactions comprised of people, organizations, data infrastructure, and technologies in service of the creatives who do great science, and the public that depends on it.

Universities will still play a critical role in this, as they should in developing and nurturing scientific talent and being a haven for broad, exploratory scientific inquiry. But the institutional inertia created by universities, publishers, and a federal funding framework needs to change so the system can adequately monitor, acknowledge, and tackle the inefficiencies it creates.

There's overlap with what I'm describing here and different schools of thought around the Open Science movement. I don't intend for this to be a replacement or substitute, but an execution focused lens on a similar problem: opening up science in the digital age.

My hope as I continue to flesh out these thoughts in future essays is to explore questions like:

• What does it mean, in practice, to share findings more frequently, outside of a narrative format (what I refer to as "knowledge units")?
• Relatedly, what should the fundamental unit of science be (is it a paper, or something smaller)?
• How do we ensure that we maintain things like attribution and impact while promoting faster experimentation and sharing of results?
• How would this vary by discipline (e.g. data-intensive vs non-data intensive fields)?
• What's a reasonable path to adoption for what I'm describing, and what do successful outcomes actually look like in the short and long term?
• How do we preserve confidentiality and invention priority without compromising the rate of progress and collaboration?
• How should we think about data interoperability to foster collaboration (especially as companies try to create lock-in through proprietary data formats during this period)?
• Progress can be measured by depth and breadth of our advancement in knowledge. Does more open science come at the cost of breadth (see here for one example)? If so, might there be a different, hybrid solution to bridge the types of research suited for academia, industry, and some team-based science in between?

...and much more, to create a timely, big picture view of the system while highlighting points of leverage for actionable change.

If you're interested in exploring some of these ideas, or are building toward the future of science — as a researcher, technologist, funder, policy lover, or dreamer — would love to chat: [email protected]

Thanks to Philip and Eamon for sanity checking this essay, and to anyone else on the internet who calls out something dumb I wrote :)!