agnes/otwarchive-sunsetarchive
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

uh forgot to update this lol

Browse source
This commit is contained in:
aggie 2026-08-13 04:49:43 +00:00
parent e96c08cdf5
commit 36c7e918c3
17 changed files with 175 additions and 27 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
.gitignore vendored
View file

@ -1,3 +1,8 @@
multi-master.info
awsliv2.zip
/aws
/aws/
/aws/*
*.swp
.DS_Store
.bundle

2
app/controllers/collection_participants_controller.rb
View file

@ -41,7 +41,7 @@ class CollectionParticipantsController < ApplicationController
def join
unless @collection
flash[:error] = t('no_collection', default: "Which collection did you want to join?")
redirect_to(request.env["HTTP_REFERER"] || root_path) and return
redirect_back_or_to root_path and return
end
participants = CollectionParticipant.in_collection(@collection).for_user(current_user) unless current_user.nil?
if participants.empty?

87
app/models/story_parser.rb
View file

@ -39,7 +39,7 @@ class StoryParser
# places for which we have a custom parse_story_from_[source] method
# for getting information out of the downloaded text
KNOWN_STORY_PARSERS = %w[deviantart dw lj].freeze
KNOWN_STORY_PARSERS = %w[ao3 deviantart dw lj].freeze
# places for which we have a custom parse_author_from_[source] method
# which returns an external_author object including an email address
@ -54,6 +54,7 @@ class StoryParser
CHAPTERED_STORY_LOCATIONS = %w[ffnet thearchive_net efiction quotev].freeze
# regular expressions to match against the URLS
SOURCE_AO3 = '(archiveofourown\.org|ao3\.org)'.freeze
SOURCE_LJ = '((live|dead|insane)journal\.com)|journalfen(\.net|\.com)|dreamwidth\.org'.freeze
SOURCE_DW = 'dreamwidth\.org'.freeze
SOURCE_FFNET = '(^|[^A-Za-z0-9-])fanfiction\.net'.freeze
@ -736,7 +737,91 @@ class StoryParser
work_params
end
# _story: the raw HTML string passed in by the importer (used only as a last-resort fallback)
# detect_tags: when true, also scrape metadata (rating, warnings, fandoms, etc.) from the work page
def parse_story_from_ao3(_story, detect_tags = true)
work_params = { chapter_attributes: {} }
# Title: prefer the visible <h2> heading on the page over the browser <title> tag.
# The <title> tag appends " [Archive of Our Own]" which we strip out with a regex,
# but the h2.title.heading is cleaner and always present on a valid work page.
title_node = @doc.at_css('h2.title.heading')
work_params[:title] = if title_node
title_node.inner_text.strip
else
# Fallback: strip the site suffix from the <title> tag (e.g. "My Fic [Archive of Our Own]" → "My Fic")
@doc.at_css('title')&.inner_text&.sub(/\s*\[Archive of Our Own\]\s*$/i, '')&.strip.to_s
end
# Summary: the blockquote inside .summary.module holds the author-written work summary.
# clean_storytext sanitizes the HTML (strips dangerous tags, normalizes whitespace, etc.)
summary_node = @doc.at_css('.summary.module blockquote.userstuff')
work_params[:summary] = clean_storytext(summary_node.inner_html) if summary_node
# Author's beginning notes: scoped inside .preface.group so we don't accidentally
# grab end notes, which live in a separate .notes.module outside the preface.
preface = @doc.at_css('.preface.group')
if preface
notes_node = preface.at_css('.notes.module blockquote.userstuff')
work_params[:notes] = clean_storytext(notes_node.inner_html) if notes_node
end
# Story text: #chapters contains the actual chapter content. We target .userstuff
# inside it to get only the prose and not chapter headings, chapter navigation, etc.
# Without this, the fallback parser would grab the entire <body>, including AO3's
# full site navigation, header, login forms, and footer.
chapters_div = @doc.at_css('#chapters')
if chapters_div
userstuff = chapters_div.at_css('.userstuff')
# If .userstuff is missing for some reason, fall back to the whole #chapters div
storytext = userstuff ? userstuff.inner_html : chapters_div.inner_html
else
# Last resort: use the raw <body> HTML, or the original _story string if even that is missing
storytext = @doc.at_css('body')&.inner_html || _story
end
work_params[:chapter_attributes][:content] = clean_storytext(storytext)
# Tag metadata lives in a <dl class="work meta group"> on every AO3 work page.
# Each tag category is a <dd> with a specific class, containing <li><a class="tag"> items.
# We map each <a> to its text, then pass it to the existing OTWA tag-processing helpers.
if detect_tags
meta_group = @doc.at_css('dl.work.meta.group')
if meta_group
# Rating is a single value (e.g. "Teen And Up Audiences"); convert_rating_string maps
# to the OTWA internal rating constant
rating = meta_group.css('dd.rating.tags li a.tag').map { |a| a.inner_text.strip }
work_params[:rating_string] = convert_rating_string(rating.first) if rating.any?
# Archive warnings (e.g. "No Archive Warnings Apply", "Graphic Depictions Of Violence")
warnings = meta_group.css('dd.warning.tags li a.tag').map { |a| a.inner_text.strip }
work_params[:archive_warning_string] = warnings.join(', ') if warnings.any?
# Fandoms, relationships, characters, and freeform tags can all be multi-value.
# DELIMITER_FOR_OUTPUT is the separator OTWA uses internally between tag values (typically ", ").
# clean_tags normalizes capitalization and strips any characters invalid for tag names.
fandoms = meta_group.css('dd.fandom.tags li a.tag').map { |a| a.inner_text.strip }
work_params[:fandom_string] = clean_tags(fandoms.join(ArchiveConfig.DELIMITER_FOR_OUTPUT)) if fandoms.any?
relationships = meta_group.css('dd.relationship.tags li a.tag').map { |a| a.inner_text.strip }
work_params[:relationship_string] = clean_tags(relationships.join(ArchiveConfig.DELIMITER_FOR_OUTPUT)) if relationships.any?
characters = meta_group.css('dd.character.tags li a.tag').map { |a| a.inner_text.strip }
work_params[:character_string] = clean_tags(characters.join(ArchiveConfig.DELIMITER_FOR_OUTPUT)) if characters.any?
# Freeform tags are the author's custom tags (the "Additional Tags" field on AO3)
freeforms = meta_group.css('dd.freeform.tags li a.tag').map { |a| a.inner_text.strip }
work_params[:freeform_string] = clean_tags(freeforms.join(ArchiveConfig.DELIMITER_FOR_OUTPUT)) if freeforms.any?
# Use the AO3 publish date as the OTWA revised_at timestamp
published = meta_group.at_css('dd.published')
work_params[:revised_at] = convert_revised_at(published.inner_text.strip) if published
end
end
# post_process_meta applies final OTWA-side transformations to work_params
# (e.g. resolving tag objects, setting defaults) before the work is saved
post_process_meta(work_params)
end
# Move and/or copy any meta attributes that need to be on the chapter rather
# than on the work itself
def shift_chapter_attributes(work_params)

3
app/views/collections/_collection_blurb.html.erb
View file

@ -77,8 +77,7 @@
data: {confirm: ts('Are you certain you want to leave this collection?')},
:method => :delete %></li>
<% else %>
<%= button_to ts("Join"), join_collection_participants_path(collection), method: :post %>
<% end %>
<%= link_to ts("Join"), join_collection_participants_path(collection) %> <% end %>
</li>
<% end %>
</ul>

11
app/views/collections/_header.html.erb
View file

@ -22,16 +22,7 @@
<%= render "challenge/#{challenge_class_name(@collection)}/challenge_navigation_user" %>
<% end %>
<% unless @collection.challenge || @collection.user_is_owner?(current_user) %>
<% if (@participant ||= @collection.get_participants_for_user(current_user).first) %>
<li><%= link_to ts("Leave"), collection_participant_path(@collection, @participant),
data: { confirm: ts("Are you certain you want to leave this collection?") },
method: :delete %></li>
<% elsif logged_in? && @collection.moderated? %>
<li><%= button_to ts("Join"), join_collection_participants_path(@collection), method: :post %></li>
<% end %>
<% end %>
<% if @collection.user_is_maintainer?(current_user) %>
<li><%= link_to ts("Membership"), collection_participants_path(@collection) %></li>
<li><%= link_to ts("Manage Items"), collection_items_path(@collection) %></li>

3
app/views/home/about.html.erb
View file

@ -3,7 +3,8 @@
<div class="userstuff">
Sunset is a website established in 2025 dedicated to hosting F/F, NB/F and NB/NB fanworks. It holds pro-freedom of fiction, queer and trans inclusive, anti-bigotry and generally compassionate ideals. We're here to have fun and love fictional characters. Sunset is run by one person, <a href="https://kissing.computer/">Agnes the Alien</a>, who dreamed this up in zher bedroom. Please be nice to zher!</p><p>THIS IS AN 18+ ARCHIVE. Sorry :(</p><p>
I'd like to shout out the websites who helped me make Sunset possible, other otwarchive instances: <a href="https://squidgeworld.org/">Squidgeworld</a>, <a href="https://superlove.sayitditto.net/">Superlove</a>, <a href="https://adastrafanfic.com/">Ad Astra</a>, and <a href="https://cfaarchive.com/">Comic Fanfiction Authors Archive</a>.</p> <br> You can link to us with the following button! <br><img src="https://file.garden/Zw17vw8ctXTQw7PV/buttonforsunset.png"> <br><br>
You can read our content policy <a href="https://sunset.femslash.club/content">here</a>.
You can read our content policy <a href="https://sunset.femslash.club/content">here</a>.<br><br>
Our logo was made by <a href="https://vgen.co/Burgirl">Burgirl on VGen!</a>
<hr><br>
<div id="femslashfans">
<a href="https://alien.hospital/femslashring"> | <img src="https://file.garden/Zw17vw8ctXTQw7PV/femslashfansonline.png"></a> | <br>

2
app/views/menu/_menu_othersites.html.erb
View file

@ -5,4 +5,4 @@
</li><li> <a href="https://adastrafanfic.com">Ad Astra</a>
</li><li><a href="https://squidgeworld.org/">SquidgeWorld</a>
</li><li> <a href="https://www.cfaarchive.org/">CFAA</a>
</li></ul>
</li><li><a href="https://fanfiction.lol">Fanfiction.LOL</a></li></ul>

6
app/views/works/edit.html.erb
View file

@ -19,11 +19,7 @@
<li><%= link_to ts('Manage Chapters'), manage_work_chapters_path(@work) %></li>
<% end %>
<% unless (@work.pseuds - current_user.pseuds).empty? %>
<li><%= button_to ts("Remove Me As Co-Creator"),
remove_user_creatorship_work_path(@work),
data: { confirm: "This will remove you from all chapters as well. Are you sure?" },
method: :patch %></li>
<% end %>
<li><%= link_to ts("Remove Me As Co-Creator"), { :action => "edit", :remove => "me"}, data: { confirm: "This will remove you from all chapters as well. Are you sure?" } %></li> <% end %>
<% if @work.posted? %>
<li><%= link_to ts('Orphan Work'), {:controller => 'orphans', :action => 'new', :work_id => @work.id} %></li>
<% end %>

33
app/views/works/edit.html.erb.save Executable file
View file

@ -0,0 +1,33 @@
<!--Descriptive page name, messages and instructions-->
<h2 class="heading"><%= ts('Edit Work') %></h2>
<%= error_messages_for :work %>
<!--/descriptions-->
<!--subnav-->
<ul class="navigation actions" role="menu">
<li><%= link_to ts('Add Chapter'), new_work_chapter_path(@work) %></li>
<% if @chapters %>
<li><%= ts('Edit Chapter:') %>
<% for chapter in @chapters %>
<% if chapter.posted? %>
<%= link_to h(chapter.position), [:edit, @work, chapter] %>
<% else %>
<%= link_to h(chapter.position) + ts(" (Draft)"), [:edit, @work, chapter] %>
<% end %>
<% end %></li>
<li><%= link_to ts('Manage Chapters'), manage_work_chapters_path(@work) %></li>
<% end %>
<% unless (@work.pseuds - current_user.pseuds).empty? %>
<li><%= link_to ts("Remove Me As Co-Creator"), { :action => "edit", :remove => "me"},
data: { confirm: "This will remove you from all chapters as well. Are you sure?" } %></li> <% end %>
<% if @work.posted? %>
<li><%= link_to ts('Orphan Work'), {:controller => 'orphans', :action => 'new', :work_id => @work.id} %></li>
<% end %>
<li><%= link_to ts("Delete Work"), confirm_delete_work_path(@work), data: {confirm: ts("Are you sure you want to delete this work? This will destroy all comments and kudos on this work as well and CANNOT BE UNDONE!")} %></li>
</ul>
<!--/subnav-->
<!--main content-->
<%= render :partial => 'standard_form' %>
<!--/content-->

BIN
awscliv2.zip Normal file
View file

Binary file not shown.

38
backup.sh Executable file
View file

@ -0,0 +1,38 @@
#!/bin/bash
cd /home/otwarchive/otwarchive || exit 1
FILE="/tmp/otwarchive_$(date +%Y-%m-%d).sql"
#sudo docker compose exec -T db \
# mysqldump -uroot \
#--databases otwarchive_production \
#--skip-comments \
#| gzip > "$FILE"
#sudo docker compose exec db mysqldump -uroot -p$MYSQL_ROOT_PASSWORD --databases otwarchive_production --skip-comments > ~/otwa/otwarchive_$(date +%Y-%m-%d_%H-%M-%S).sql.gz
#
#touch ~/otwabck/otwarchive_$(date +%Y-%m-%d_%H-%M-%S).sql
#touch "/tmp/otwarchive_$(date +%Y-%m-%d).sql"
sudo docker compose exec -T db sh -c \
'mysqldump -uroot -p"$MYSQL_ROOT_PASSWORD" --databases otwarchive_production --skip-comments' \
| gzip > "/tmp/otwarchive_$(date +%Y-%m-%d).sql"
if [ ${PIPESTATUS[0]} -ne 0 ]; then
echo "MySQL dump failed"
rm -f "$FILE"
exit 1
fi
aws --endpoint-url https://t3.storage.dev \
s3 cp "$FILE" \
"s3://sunsetti/mysql/otwarchive/$(basename "$FILE")"
if [ $? -ne 0 ]; then
echo "S3 upload failed"
rm -f "$FILE"
exit 1
fi
rm -f "$FILE"
echo "Backup completed successfully"

10
config/locales/mailers/en.yml
View file

@ -533,7 +533,7 @@ en:
invitation:
been_invited: You've been invited to join Sunset!
features: With an account, you can post fanworks, use bookmarks to keep track of works you enjoyed, receive subscription emails when your favorite creators or works update, customize the way the site looks for you, and more!
has_invited: "%{user_name} has invited you to join the Archive of Our Own!"
has_invited: "%{user_name} has invited you to join Sunset Archive!"
html:
about: Sunset is a free, noncommercial archive built by and for fans.
activation_support: After you sign up, you'll receive an account activation email. If you do not receive this email after 48 hours, please %{support_link}.
@ -627,9 +627,9 @@ en:
subject: "[%{app_name}][%{collection_title}] Potential assignment generation complete"
recipient_notification:
collection:
html: A gift work has been posted for you in the %{collection_link} collection at the Archive of Our Own!
text: A gift work has been posted for you in the "%{collection_title}" collection (%{collection_url}) at the Archive of Our Own!
no_collection: A gift work has been posted for you at the Archive of Our Own!
html: A gift work has been posted for you in the %{collection_link} collection at Sunset Archive!
text: A gift work has been posted for you in the "%{collection_title}" collection (%{collection_url}) at the Sunset Archive!
no_collection: A gift work has been posted for you at Sunset!
subject:
collection: "[%{app_name}][%{collection_title}] A gift work for you from %{collection_title}"
no_collection: "[%{app_name}] A gift work for you"
@ -659,7 +659,7 @@ en:
html: There's lots of information and advice on how to use the Archive in our %{faq_link}. You'll find the latest news about site developments on %{admin_posts_link}. If you need more help, run into a bug, or have questions or comments, please %{contact_support_link}, who are always happy to help out.
text: 'There''s lots of information and advice on how to use the Archive in our FAQ at %{faq_url}. You''ll find the latest news about site developments on Sunset News at %{admin_posts_url}. If you need more help, run into a bug, or have questions or comments, please contact Support, who are always happy to help out: %{contact_support_url}.'
subject: "[%{app_name}] Activate your account"
welcome: Welcome to the Archive of Our Own, %{login}!
welcome: Welcome to Sunset, %{login}!
users:
mailer:
confirmation_instructions:

2
config/locales/views/en.yml
View file

@ -2322,7 +2322,7 @@ en:
agreement_confirm: Yes, I have read the Terms of Service, including the Content Policy and Privacy Policy, and agree to them.
agreement_required_html: Before you begin using Sunset, you must agree to our %{terms_of_service_link}, including the %{content_policy_link} and %{privacy_policy_link}.
content_policy: Content Policy (opens in new window)
data_processing_confirm: By checking this box, you consent to the processing of your personal data in the United States and other jurisdictions in connection with our provision of AO3 and its related services to you. You acknowledge that the data privacy laws of such jurisdictions may differ from those provided in your jurisdiction. For more information about how your personal data will be processed, please refer to our Privacy Policy.
data_processing_confirm: By checking this box, you consent to the processing of your personal data in the United States and other jurisdictions in connection with our provision of Sunset and its related services to you. You acknowledge that the data privacy laws of such jurisdictions may differ from those provided in your jurisdiction. For more information about how your personal data will be processed, please refer to our Privacy Policy.
over_thirteen_confirm: Yes, I am at least 18.
over_thirteen_required: You need to be at least 18 years old to become a registered member of the Archive. (Sorry to anyone younger! You'll be more than welcome when the time comes.)
privacy_policy: Privacy Policy (opens in new window)

BIN
public/images/ao3_logos/logo.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 113 KiB

After

Width:  |  Height:  |  Size: 52 KiB

BIN
public/images/ao3_logos/logo_42.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 113 KiB

After

Width:  |  Height:  |  Size: 52 KiB

BIN
public/images/logo.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 113 KiB

After

Width:  |  Height:  |  Size: 52 KiB

BIN
public/images/logo_42.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 113 KiB

After

Width:  |  Height:  |  Size: 52 KiB