.png)
4 months ago
8
When I am using an online Jupyter notebooks (e.g., DeepLearning.AI, Colab, Kaggle, etc.), I usually prefer to be able to reproduce it locally. I want this mainly so I can easily take snippets of Python code and incorporate them into a proper script or application outside of a notebook.
The hard part is knowing the proper version of Python and all the packages that the notebook uses. When using an online notebook, this isn't always obvious. Python, is infamous for its ability to place you into "dependency hell". Anything you use online is bound to be out-of-date very soon, so you really need to know the exact version of everything.
So I wrote this little bit of code that I paste into a notebook cell and run. It tells me what I need to do to (a) install Python, (b) set up a virtual environment and (c) specify a requirements.txt file that I can then pip install.
Here it is:
# Paste this into a Jupyter notebook cell and run it; then follow the outputted
# instructions.
import subprocess
import sys
print("How to create a virtual environment to match this notebook:\n")
version = f"{sys.version_info.major}.{sys.version_info.minor}"
print(f"Install python version: {version}\n")
print(f" + On Mac: 'brew install python@{version}'")
print(f" + Other OS: ask your favorite AI or Google it.\n")
print(f"Run 'python{version} -m venv .venv' to create a new virtual environment.")
print("Run 'source .venv/bin/activate' to activate your virtual environment.\n")
print("Put these entries in a 'requirements.txt' file and run")
print(" 'pip install -r requirements.txt'.\n")
print("#=========== Begin requirements.txt ===========")
result = subprocess.run([sys.executable, '-m', 'pip', 'freeze'], capture_output=True, text=True)
lines = result.stdout.strip().split('\n')
clean_requirements = [] # strip out @ file:/// refs since won't be in your local env
for line in lines:
if '@ file://' not in line and line.strip():
clean_requirements.append(line)
print('\n'.join(clean_requirements))
print("#============ End requirements.txt ============")
Here's sample output:
How to create a virtual environment to match this notebook: Install python version: 3.11 + On Mac: 'brew install [email protected]' + Other OS: ask your favorite AI or Google it. Run 'python3.11 -m venv .venv' to create a new virtual environment. Run 'source .venv/bin/activate' to activate your virtual environment. Put these entries in a 'requirements.txt' file and run 'pip install -r requirements.txt'. #=========== Begin requirements.txt =========== acp-sdk==0.10.1 aiohappyeyeballs==2.6.1 aiohttp==3.12.12 aiosignal==1.3.2 alembic==1.16.1 annotated-types==0.7.0 anyio==4.9.0 appdirs==1.4.4 argon2-cffi==25.1.0 argon2-cffi-bindings==21.2.0 arrow==1.3.0 asgiref==3.8.1 asttokens==3.0.0 async-lru==2.0.5 attrs==25.3.0 auth0-python==4.10.0 babel==2.17.0 backoff==2.2.1 bcrypt==4.3.0 beautifulsoup4==4.13.4 bleach==6.2.0 blinker==1.9.0 build==1.2.2.post1 cachetools==5.5.2 certifi==2025.4.26 cffi==1.17.1 charset-normalizer==3.4.2 chroma-hnswlib==0.7.6 chromadb==0.5.23 click==8.2.1 cohere==5.15.0 colorama==0.4.6 coloredlogs==15.0.1 comm==0.2.2 crewai==0.120.1 crewai-tools==0.45.0 cryptography==45.0.4 dataclasses-json==0.6.7 debugpy==1.8.14 decorator==5.2.1 defusedxml==0.7.1 deprecation==2.1.0 distro==1.9.0 dnspython==2.7.0 docker==7.1.0 docstring_parser==0.16 durationpy==0.10 email_validator==2.2.0 embedchain==0.1.128 et_xmlfile==2.0.0 executing==2.2.0 fastapi==0.115.12 fastapi-cli==0.0.7 fastavro==1.11.1 fastjsonschema==2.21.1 filelock==3.18.0 flatbuffers==25.2.10 fqdn==1.5.1 frozenlist==1.7.0 fsspec==2025.5.1 google-auth==2.40.3 googleapis-common-protos==1.70.0 gptcache==0.1.44 greenlet==3.2.3 grpcio==1.73.0 h11==0.16.0 h2==4.2.0 hf-xet==1.1.3 hpack==4.1.0 httpcore==1.0.9 httptools==0.6.4 httpx==0.28.1 httpx-sse==0.4.0 huggingface-hub==0.33.0 humanfriendly==10.0 hyperframe==6.1.0 idna==3.10 importlib_metadata==8.7.0 importlib_resources==6.5.2 instructor==1.8.3 ipykernel==6.29.5 ipython==9.3.0 ipython_pygments_lexers==1.1.1 ipywidgets==8.1.7 isoduration==20.11.0 janus==2.0.0 jedi==0.19.2 Jinja2==3.1.6 jiter==0.8.2 json5==0.12.0 json_repair==0.46.2 jsonpatch==1.33 jsonpickle==4.1.1 jsonpointer==3.0.0 jsonref==1.1.0 jsonschema==4.24.0 jsonschema-specifications==2025.4.1 jupyter==1.1.1 jupyter-console==6.6.3 jupyter-events==0.12.0 jupyter-lsp==2.2.5 jupyter_client==8.6.3 jupyter_core==5.8.1 jupyter_server==2.16.0 jupyter_server_terminals==0.4.4 jupyterlab==4.4.3 jupyterlab_pygments==0.3.0 jupyterlab_server==2.27.3 jupyterlab_widgets==3.0.15 kubernetes==33.1.0 lancedb==0.23.0 langchain==0.3.25 langchain-cohere==0.3.5 langchain-community==0.3.25 langchain-core==0.3.65 langchain-experimental==0.3.4 langchain-openai==0.2.14 langchain-text-splitters==0.3.8 langsmith==0.3.45 litellm==1.68.0 load-dotenv==0.1.0 Mako==1.3.10 markdown-it-py==3.0.0 MarkupSafe==3.0.2 marshmallow==3.26.1 matplotlib-inline==0.1.7 mdurl==0.1.2 mem0ai==0.1.107 mistune==3.1.3 mmh3==5.1.0 monotonic==1.6 mpmath==1.3.0 multidict==6.4.4 mypy_extensions==1.1.0 nbclient==0.10.2 nbconvert==7.16.6 nbformat==5.10.4 nest-asyncio==1.6.0 networkx==3.5 nodeenv==1.9.1 notebook==7.4.3 notebook_shim==0.2.4 numpy==2.3.0 oauthlib==3.2.2 onnxruntime==1.22.0 openai==1.75.0 openpyxl==3.1.5 opentelemetry-api==1.34.1 opentelemetry-exporter-otlp-proto-common==1.34.1 opentelemetry-exporter-otlp-proto-grpc==1.34.1 opentelemetry-exporter-otlp-proto-http==1.34.1 opentelemetry-instrumentation==0.55b1 opentelemetry-instrumentation-asgi==0.55b1 opentelemetry-instrumentation-fastapi==0.55b1 opentelemetry-instrumentation-httpx==0.55b1 opentelemetry-proto==1.34.1 opentelemetry-sdk==1.34.1 opentelemetry-semantic-conventions==0.55b1 opentelemetry-util-http==0.55b1 orjson==3.10.18 overrides==7.7.0 packaging==24.2 pandas==2.3.0 pandocfilters==1.5.1 parso==0.8.4 pdfminer.six==20250327 pdfplumber==0.11.6 pexpect==4.9.0 pillow==11.2.1 platformdirs==4.3.8 portalocker==2.10.1 posthog==3.25.0 prometheus_client==0.22.1 prompt_toolkit==3.0.51 propcache==0.3.2 protobuf==5.29.5 psutil==7.0.0 psycopg==3.2.9 psycopg-binary==3.2.9 ptyprocess==0.7.0 pure_eval==0.2.3 pyarrow==20.0.0 pyasn1==0.6.1 pyasn1_modules==0.4.2 pycparser==2.22 pydantic==2.11.5 pydantic-settings==2.9.1 pydantic_core==2.33.2 Pygments==2.19.1 PyJWT==2.10.1 pypdf==5.6.0 pypdfium2==4.30.1 PyPika==0.48.9 pyproject_hooks==1.2.0 pyright==1.1.402 pysbd==0.3.4 python-dateutil==2.9.0.post0 python-dotenv==1.1.0 python-json-logger==3.3.0 python-multipart==0.0.20 pytube==15.0.0 pytz==2025.2 pyvis==0.3.2 PyYAML==6.0.2 pyzmq==27.0.0 qdrant-client==1.14.2 redis==6.2.0 referencing==0.36.2 regex==2024.11.6 requests==2.32.4 requests-oauthlib==2.0.0 requests-toolbelt==1.0.0 rfc3339-validator==0.1.4 rfc3986-validator==0.1.1 rich==13.9.4 rich-toolkit==0.14.7 rpds-py==0.25.1 rsa==4.9.1 schema==0.7.7 Send2Trash==1.8.3 shellingham==1.5.4 six==1.17.0 smolagents==1.18.0 sniffio==1.3.1 soupsieve==2.7 SQLAlchemy==2.0.41 stack-data==0.6.3 starlette==0.46.2 sympy==1.14.0 tabulate==0.9.0 tenacity==9.1.2 terminado==0.18.1 tiktoken==0.9.0 tinycss2==1.4.0 tokenizers==0.20.3 tomli==2.2.1 tomli_w==1.2.0 tornado==6.5.1 tqdm==4.67.1 traitlets==5.14.3 typer==0.16.0 types-python-dateutil==2.9.0.20250516 types-requests==2.32.4.20250611 typing-inspect==0.9.0 typing-inspection==0.4.1 typing_extensions==4.14.0 tzdata==2025.2 uri-template==1.3.0 urllib3==2.4.0 uv==0.7.12 uvicorn==0.34.3 uvloop==0.21.0 watchfiles==1.0.5 wcwidth==0.2.13 webcolors==24.11.1 webencodings==0.5.1 websocket-client==1.8.0 websockets==15.0.1 widgetsnbextension==4.0.14 wrapt==1.17.2 yarl==1.20.1 zipp==3.23.0 zstandard==0.23.0 #============ End requirements.txt ============
