suggested updates to crawler.py

[raybellwaves]

Cool project!

I wanted to start small and understand what this project does.

I updated crawler.py here https://github.com/raybellwaves/asap_pdf/blob/uv-inline-deps/python_components/crawler/crawler.py

I didn't bother making a PR as this is mostly developer choice.

I used uv inline deps so anyone can run the code as it (with https://docs.astral.sh/uv/ installed). e.g.

uv run https://raw.github.com/raybellwaves/asap_pdf/refs/heads/uv-inline-deps/python_components/crawler/crawler.py \
    https://www.whitman.edu/academics/departments-and-programs/mathematics \
    output.csv \
    --depth 1

For full transparency I did use AI to help: https://claude.ai/share/a5534af1-24ed-4c18-a04d-7747b06254b0

It identified things like unused imports but you also get those from running https://docs.astral.sh/ruff/

Keep up the great work!

[lkacenja]

Hi @raybellwaves, thank for the interest in our app. We are so excited for others to get value from it. We have somewhat limited experience with uv, but it generally seems great. I will have to try out what you've suggested here. If you are interested, we'd love to set up a quick call with you and/or your team. We'd be eager to learn how we could improve the app and what problems Maryland is solving with AI. Thanks again!