We’re taking canonical Twitter URLs back, one archive at a time:
36 replies
@zachleat dont you ever sleep ;)
or is it the post-musk-twitter-rush thats kicking in ?
@zachleat If only I could request my archives 😭 It's an endless loop where I need to put in my password and then get a verification token from an email but my guess is a shut off microservice sends that email so I never get the code
@zachleat it might be a good idea to separate the implementation from the database. This way other implementations in other languages can use the same canonical list.
@dominik Great idea! Filed this here https://github.com/tweetback/tweetback-canonical/issues/2
@zachleat seoh my gosh how do you even think of these things
@danleatherman brain do stuff, sometimes
@zachleat ooh, I was thinking about building something to turn my Twitter export into a useful website, and now I see you're already on the case! Nice. I shall use that :)
@sil yeah—working with @nhoizey on this!
you can see a few previews I’ve shipped here:
https://www.zachleat.com/twitter/
https://twitter.11ty.dev
@zachleat Cool! Did you redownload all your tweets yourself? Because the archives has many issues... Did you pay for an API token?
@FremyCompany I used the archive as a baseline and then repopulate from the API—I wasn’t aware API tokens costed money 👀
@zachleat Oh, I don't know if they cost anything, I'm just asking for your experience here to learn more about what you did.
Do you care for instance about your retweets, or just your own tweets?
I find that backing up your retweets / quoted tweets / likes is the most difficult, I was wondering if you had any thoughts on this already.
@FremyCompany I do back up retweets, although the content is truncated e.g. https://www.zachleat.com/twitter/1587076514939641856/
quote tweets link directly to the source https://www.zachleat.com/twitter/1586436147386888193/
My concern with the retweet thing is that they don’t really obey norms around upstream deletes
@zachleat Yes, because of the truncation (which often deletes the attached media and expanded urls), I am currently recrawling all of them myself (masquerading as the twitter website on my PC).
@zachleat But for now, I'm only thinking about keeping that data private, but it's interesting to think about the issue of keeping backups of other people's messages that they can't delete easily.
I will keep this in mind if I want to host this later on, thanks for the thought!
Any other challenge like this you encountered?
@FremyCompany maybe we should collab on tweetback! We have about 8 folks on the org right now
cc @nhoizey
@zachleat I think we should open the repository ASAP, and issues and PRs will be a good way to discuss such topics.
@fremycompany@nhoizey @FremyCompany huge upvote—I think it’s ready enough for early previews
@zachleat I think so!
You can see this in action on the Eleventy Twitter Archive, where any links to https://twitter.com/zachleat/ now link to https://zachleat.com/twitter/ instead.
@zachleat This is awesome. Now can you build something that will preserve all the t.co shortened links that will disappear when the whole thing goes whale-up?
@zero_gravitas tweetback archives will get rid of t.co links in your content when they build! 🏆
@zachleat thanks for doing this. Zach. I was planning to learn 11ty (mostly Hugo at the moment) to put together my archives so this is most helpful
@skinnylatte yeah! Stay tuned! We will have a starter project here for this in a bit!
@zachleat You may also want to consider transforming all twitter links in your archive to nitter https://nitter.snopyta.org/zachleat, for example rather than linking to https://twitter.com/buildWithLit/status/1591173922678079489 you can link to https://nitter.snopyta.org/buildWithLit/status/1591173922678079489 details about nitter https://nitter.snopyta.org/about, this can act as a fallback if they haven't created their own archive
@thomasreggi ooh can you file an issue on the repo for this? I think it’s worth discussing
-
@thomasreggi Thank you!!
-