agnes/dw-commentfest-scrape
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
This scrapes Dreamwidth comments on a specific posts and sends them to a Discord webhook, generating the comment text and a link to the comment in HTML format
5 commits 1 branch 0 tags 39 KiB
PowerShell 63.4%
Shell 21.9%
Python 14.7%
Find a file
agnes bdca58a85b Update scraper.py 
accidentally doxxa my webhook url so i changed it
2026-06-15 00:28:00 +00:00
.venv/bin first 2026-06-14 19:48:45 -04:00
bin first 2026-06-14 19:48:45 -04:00
.gitignore first 2026-06-14 19:48:45 -04:00
last_run.json first 2026-06-14 19:48:45 -04:00
pyvenv.cfg.example fixa some stuff 2026-06-14 19:54:11 -04:00
README.md readme fix 2026-06-14 20:15:37 -04:00
scrape5hr.service fixa some stuff 2026-06-14 19:54:11 -04:00
scrape5hr.timer first 2026-06-14 19:48:45 -04:00
scraper.py Update scraper.py 2026-06-15 00:28:00 +00:00

README.md

This scrapes Dreamwidth comments on a specific posts and sends them to a Discord webhook, generating the comment text and a link to the comment in HTML format <a href="url">text</a> wrapped in backticks. When put on systemd it runs every five hours (don't change that, to be respectful to DW) and if no new comments have been posted since the last one then it returns no comments. this is my first python script idk

How to use

Follow the comments in the scraper.py code to edit the code appropriately for your usage.

cp pyvenv.cfg.example pyvenv.cfg

replace in that file with your username

REGARDING COMMENT PAGES

You will have to manually change the url for the comment page when new comment pages appear. I know, sorry.

example https://toothpastejuice.dreamwidth.org/2061.html?view=flat&page=3#comments