Tech Talk: Frontpage link aggregator
Our first tech talk is by Tom Sherman, creator of the Frontpage link aggregator app built on ATProtocol.
Welcome to ATProtocol Development's first tech talk, and thank you to Tom Sherman from Frontpage for being our first presenter.
Join Tom for a presentation about building Frontpage on ATProtocol: Making a Lexicon for link aggregation and the experience of working with the protocol. Tom will present and we'll have time for live discussion and Q&A. This is a technical talk, so we expect some familiarity with ATProtocol concepts, but all are welcome, including users curious about new protocol apps.
You can RSVP on Smoke Signal, an ATProtocol social events protocol. We'll be hosting a Smoke Signal tech talk the week after. The calendar page here has both a link to our Luma as well as an embedded Smoke Signal calendar owned by the @atprotocol.dev account.
Please register to join us live to get the Zoom link for Thursday, August 29th, at 9am PST, 12pm EST, 1800 CEST.
Video
Thanks to Tom and the Nor.Dev crew for building this and walking us through their process!
Here are the slides from the talk:
Plus the Excalidraw live version.
- The code for Frontpage is available in this monorepo https://github.com/likeandscribe/unravel
- PDS deployment on Fly.io is here https://github.com/likeandscribe/unravel/tree/main/packages/pds
Chatlog
00:23:40 Laurens Hof: Nice shirt tom
00:23:52 will: huge snub
00:23:57 jake.saterlay: Reacted to "huge snub" with ๐
00:24:07 Alex Scotton: Reacted to "huge snub" with ๐
00:24:10 Damien Sedgwick: Reacted to "huge snub" with ๐
00:24:12 timothy: Reacted to "huge snub" with ๐
00:25:26 Boris Mann: https://frontpage.fyi/
00:25:35 Boris Mann: You can sign in with an ATProto app password
00:26:50 Damien Sedgwick: Muted Boris
00:27:00 Paul Frazee: I see it
00:27:01 will: I see it
00:27:04 Damien Sedgwick: I can see it also
00:27:32 Boris Mann: But I tricked everyone into chiming in
00:27:47 Paul Frazee: Reacted to "But I tricked everyo..." with ๐
00:27:50 Boris Mann: Excalidraw slides, very cool
00:32:49 Paul Frazee: all tech decisions are mutable
00:35:18 Paul Frazee: drainpipe ๐
00:35:26 Ryan Barrett: firehose filtering: https://github.com/bluesky-social/atproto/discussions/2418
00:35:38 Ryan Barrett: Reacted to "drainpipe ๐" with ๐ฏ
00:37:22 Boris Mann: โevery change on HTTP -- all apps on the Internetโ
00:37:33 Ryan Barrett: Reacted to "โevery change on HTT..." with ๐
00:37:36 Boris Mann: Key โ this is what I am thinking about, what I call โproto appsโ
00:37:37 Paul Frazee: drainpipe is hitting two things that we ran into:
filtering to an appโs interested messages
interface that works with serverless (nextjs backend)
00:37:51 will: written in rust too
00:38:07 Drew: Reacted to "written in rust too" with ๐
00:38:13 Paul Frazee: lot of app devs are nextjs/serverless brained rn so solving that interface issue for serverless is quite clutch
00:38:15 Boris Mann: Weโll get GitHub links for the post
00:38:32 Paul Frazee: do people still say clutch
00:38:46 Ryan Barrett: elder millenials unite
00:38:49 timothy: https://github.com/likeandscribe/unravel
00:38:49 Boris Mann: Stop trying to make clutch a thing, Paul
00:39:03 will: https://github.com/likeandscribe/unravel
https://github.com/likeandscribe/unravel/tree/main/packages-rs/drainpipe
00:39:06 Paul Frazee: Reacted to "elder millenials uni..." with โ
00:39:11 jon: Conceptually, it seems to me, (Relay + Drainpipe) = customi Relay
00:39:11 Paul Frazee: Reacted to "Stop trying to make ..." with ๐
00:39:12 Boris Mann: I also want that Fly.io stuff, including PDS on Fly
00:39:31 timothy: Itโs a monorepo, btw
00:39:37 Paul Frazee: ooooh we got an answer for you on the read-after-write lag
00:39:48 Paul Frazee: oh maybe you do too
00:39:50 Ryan Barrett: https://docs.bsky.app/docs/advanced-guides/read-after-write
00:39:56 Boris Mann: Replying to "Itโs a monorepo, btw"
Yep โฆ would like it split out. Lot of people would like to not dedicate an entire VPS to a PDS
00:39:57 Paul Frazee: yeah we got a better answer now
00:40:02 Ryan Barrett: Reacted to "yeah we got a better..." with ๐
00:40:28 Laurens Hof: Reacted to "elder millenials uni..." with โ
00:40:56 Paul Frazee: yes the two writes is the move weโre picking up
00:41:18 Laurens Hof: Ohh is this why I got errors recently when I was posting on frontpage?
00:41:47 will: errors could be just tom deleting the db as usual
00:41:55 Laurens Hof: Reacted to "errors could be just..." with ๐
00:42:30 timothy: Reacted to "errors could be just..." with ๐
00:42:35 jake.saterlay: Reacted to "errors could be just..." with ๐
00:43:31 Bryan Newbold: Reacted to "Conceptually, it s..." with ๐
00:43:46 Damien Sedgwick: Reacted to "errors could be ju..." with ๐
00:45:21 Ryan Barrett: multi-tenant
00:45:48 Boris Mann: Preferences vsโฆ.. forking?
00:46:05 Boris Mann: Going to ask about forking / extending lexicons. Composable Lexicon?
00:47:32 jon: Rudy, these two paths to community remind me of your early thoughts on a BlackSky community in โThe Moderation Postโ โฆ you didnโt have an AppView on your list of what might be needed, and I remember thinking โhmm I wonder if it will turn out to need an AppView as wellโ
00:49:07 Laurens Hof: Reacted to "Rudy, these two path..." with ๐
00:49:19 Boris Mann: Bryan talked about this https://bsky.app/profile/bnewbold.net/post/3l2jhuf4jpm27
00:49:39 Paul Frazee: 2 kinds of compatibility:
interpretation of lexicons across each othersโ apps
correlation of records (establishing the canonical link) which are being used for a shared comments section
00:50:47 Laurens Hof: Bluesky mirror would help with how I try to use frontpage
00:53:00 Boris Mann: at://
00:54:16 Ryan Barrett: editing app.bsky post records doesn't hurt anything in the bsky.app client, we do it, the edits just don't show up (yet)
00:54:37 Paul Frazee: ๐
00:54:57 Boris Mann: โAtmosphereโ โ this is so NOT happening as a phrase
00:55:10 Paul Frazee: ITS HAPPENING BORIS
00:55:15 will: ๐
00:55:27 Laurens Hof: ATmosphere is so good tho
00:55:36 Nicolas Gimenez (ZKorum): ๐
00:55:42 jon: I totally disagree, Atmosphere is great!
00:55:46 Ryan Barrett: almost as bad as "fediverse"
00:55:55 Laurens Hof: My baseline is โfediverseโ, so I think everything is good
00:55:56 Ryan Barrett: ๐ฆ
00:55:57 jon: Much better thanfediverse! lol
00:56:07 Ryan Barrett: Reacted to "Much better thanfedi..." with ๐
00:56:45 jake.saterlay: wow what a brilliantly implemented avatar
00:56:54 timothy: Reacted to "wow what a brilliant..." with ๐
00:56:57 timothy: Reacted to "wow what a brilliant..." with ๐
00:57:04 will: Reacted to "wow what a brilliant..." with ๐
00:57:11 Damien Sedgwick: Reacted to "wow what a brillia..." with ๐
00:57:13 Damien Sedgwick: Reacted to "wow what a brillia..." with ๐
00:57:24 Nick Gerakines (he/him): 10,000 god help us
00:57:26 Tom: Design is definitely v0 (in more ways than one)
00:57:50 timothy: Not again, he didn't
00:57:57 Paul Frazee: I think thatโs a pretty solid design for a v0
00:58:14 Ryan Barrett: Reacted to "I think thatโs a pre..." with ๐
00:58:28 Laurens Hof: Reacted to "I think thatโs a pre..." with ๐
00:58:59 will: Reacted to "I think thatโs a pre..." with ๐
00:59:05 Nick Gerakines (he/him): Reacted to "I think thatโs a ..." with โ
00:59:10 Ryan Barrett: labels? labelers?
00:59:16 Damien Sedgwick: oh my posts being unfairly moderated.
00:59:51 jon: Replying to "labels? labelers?"
Thatโs a really interesting idea. Would it be easy to repurpose Ozone to work on Frontpage?
01:00:11 Genco: Thoughts on almost every atproto identity being owned by Bluesky PBC? Especially since you raised other centralisation concerns
01:00:21 Ryan Barrett: Replying to "labels? labelers?"
hopefully! seems like the hope with labels, labelers/mod services, etc is that they apply to the whole ecosystem and protocol, not just bsky.app
01:00:23 Alex Scotton: Replying to "oh my posts being un..."
deserved
01:00:27 jake.saterlay: Reacted to "deserved" with ๐
01:00:34 Damien Sedgwick: I actually had this problem recently on Reddit. The answer was coming up on google but clicking the link came back with 'This post has been deleted' - So frustrating!
01:00:38 timothy: Reacted to "deserved" with ๐
01:01:35 Alex Scotton: Will the Q&A be on the recording?
01:01:46 Nick Gerakines (he/him): lol w h o d o e s
01:01:49 Boris Mann: @Alex Scotton yes it will be
01:01:53 Alex Scotton: Reacted to "@Alex Scotton yes it..." with โค๏ธ
01:02:00 Damien Sedgwick: Reacted to "lol w h o d o e s" with ๐
01:02:15 Boris Mann: The DWeb Principles are a good outline what I consider DWeb / Web3 https://getdweb.net/principles/
01:02:16 Alex Scotton: Replying to "@Alex Scotton yes it..."
Awesome, thanks for hosting! Tom, this is cool - well presented
01:02:26 Damien Sedgwick: Reacted to "deserved" with ๐
01:02:34 will: wrapping my head around the whole architecture (pds etc) was the hardest part
01:02:45 jake.saterlay: Replying to "wrapping my head aro..."
+1
01:02:48 Boris Mann: Technically, I think of it as data structures, cryptography, and distributed systems
01:02:59 timothy: We made our own wireframe of the architecture and still donโt fully understand it
01:03:03 Boris Mann: Replying to "wrapping my head aro..."
Ideally this is why we do more of these. Need to do an ATproto 101
01:03:03 Alex Scotton: Replying to "wrapping my head aro..."
+2
01:03:16 Nick Gerakines (he/him): For later: Have you done any cost analysis on if severless is going to be managable in the long term with projected network growth?
01:03:16 Alex Scotton: Replying to "wrapping my head aro..."
@Boris Mann please do this.. ELI5
01:03:17 Hilary Baumann - FascinationDesign.com: Reacted to "Ideally this is why ..." with ๐
01:03:41 will: Replying to "wrapping my head aro..."
have like 3 images pinned in our discord just in case we forget ๐
01:04:24 Laurens Hof: (Side note since half the bsky team is in chat: is the notification โx followed you backโ new? Its nice)
01:04:25 Paul Frazee: weโve got a new atproto website coming up, working on a lot more intro resources
01:04:33 Hilary Baumann - FascinationDesign.com: Reacted to "weโve got a new atpr..." with ๐
01:04:35 Laurens Hof: Reacted to "weโve got a new atpr..." with ๐
01:04:40 Nick Gerakines (he/him): Reacted to "weโve got a new a..." with ๐
01:04:45 Paul Frazee: Replying to "(Side note since hal..."
yeah like, last month or two
01:04:54 Nicolas Gimenez (ZKorum): Reacted to "weโve got a new a..." with ๐
01:04:58 Boris Mann: Ryan is @snarfed and builder of Bridgy
01:04:58 will: Reacted to "weโve got a new atpr..." with ๐
01:04:59 will: Reacted to "weโve got a new atpr..." with ๐
01:05:05 Paul Frazee: Reacted to "Ryan is @snarfed and..." with ๐ฅ
01:05:10 Laurens Hof: Reacted to "yeah like, last mont..." with ๐
01:05:10 will: Reacted to "Ryan is @snarfed and..." with ๐ฅ
01:05:12 Rudy Fraser: Reacted to "weโve got a new atpr..." with ๐
01:05:59 timothy: Reacted to "weโve got a new atpr..." with ๐
01:06:00 timothy: Reacted to "weโve got a new atpr..." with ๐
01:07:44 Damien Sedgwick: we are degenerates
01:07:49 will: he's snubbed me again
01:07:50 Max Saal: Reacted to we are degenerates with "โค๏ธ"
01:07:51 Nick Gerakines (he/him): Reacted to "we are degenerates" with ๐
01:07:52 Boris Mann: Replying to "we are degenerates"
Web3
01:08:00 timothy: We play padel together
01:08:06 Genco: Replying to "Thoughts on almost e..."
all but 36565 - and 24812 of those is plc spam with pds https://uwu
01:08:10 Damien Sedgwick: Reacted to "We play padel toge..." with ๐
01:08:15 will: Reacted to "We play padel togeth..." with ๐
01:08:21 Paul Frazee: one very specific way this relay / commercial services question manifests is: how much do we want to put into the relay api (making it protocol) vs how much is separate commercial services (like drainpipe)
01:08:37 Boris Mann: Replying to "one very specific wa..."
Yes exactly
01:08:37 Ryan Barrett: Reacted to "one very specific wa..." with ๐
01:08:44 dan: Reacted to "one very specific wa..." with ๐
01:09:08 jake.saterlay: who
01:09:09 will: ๐ฅน
01:09:16 Tom: Reacted to "๐ฅน " with ๐
01:09:17 Paul Frazee: Replying to "one very specific wa..."
Iโve been in about 4 meetings in the past 2 weeks about this topic so itโs a good time to talk about this!
01:09:18 jon: Reacted to "one very specific wa..." with ๐
01:09:32 timothy: Reacted to "๐ฅน " with ๐
01:09:40 timothy: Reacted to "we are degenerates" with โค๏ธ
01:09:40 timothy: Reacted to "we are degenerates" with ๐
01:10:37 Damien Sedgwick: Do not tarnish me with your serverless brush.
01:10:42 timothy: Donโt host your psql db on Vercel on the free plan. This is why we were down recently.
01:10:48 Nick Gerakines (he/him): Good problems to have lol
01:10:57 Alex Scotton: Reacted to "Donโt host your psql..." with ๐
01:11:13 Damien Sedgwick: Reacted to "Donโt host your p..." with ๐
01:11:21 timothy: PDS stuff could probably be a GitHub template, I guess
01:11:37 Boris Mann: Reacted to "PDS stuff could prob..." with ๐
01:12:30 Ryan Barrett: Reacted to "Do not tarnish me wi..." with ๐
01:12:45 Boris Mann: Video with Bryan on the PLC Directory https://atprotocol.dev/bluesky-and-did-plc/
01:13:34 Ryan Barrett: Reacted to "Video with Bryan on ..." with ๐
01:16:04 Ryan Barrett: also Bluesky team can (and should) eventually give users on plc.directory their rotation keys. those users aren't stuck without their keys forever
01:16:55 Ryan Barrett: omg paul
01:17:06 Boris Mann: NO ONE MENTION RDF
01:17:36 Ryan Barrett: Reacted to "NO ONE MENTION RDF" with ๐ฏ
01:18:31 Nicolas Gimenez: How do we make data from different Lexicons communicate with each other? In other words Frontpage <--> bluesky integration and other appviews?
01:18:32 Paul Frazee: Replying to "NO ONE MENTION RDF"
mk ultra activation phrase for me
01:18:42 Ryan Barrett: Reacted to "How do we make data ..." with ๐
01:18:44 Laurens Hof: Reacted to "How do we make data ..." with ๐
01:18:44 Paul Frazee: Reacted to "How do we make data ..." with ๐
01:18:55 Tom: WebAuthN?
01:19:00 Boris Mann: Yeah, composability. โLinkโ type in Smoke Signal <> Frontpage
01:19:25 Boris Mann: Replying to "WebAuthN?"
Different layer. Tools on the outside. Like passkeys.
01:19:26 Genco: yes! my point was very much that what paul mentioned about being able to move from an adversarial pds is only possible when you control your keys and constantly watch the plc
01:19:28 Alex Scotton: Thanks everyone, gotta leave (I was meant to 10 mins ago, but too fascinating)
01:19:38 Tom: Reacted to "Different layer. Too..." with ๐
01:19:41 jake.saterlay: Reacted to "Thanks everyone, got..." with โค๏ธ
01:19:49 Tom: Reacted to "Thanks everyone, got..." with โค๏ธ
01:20:02 Ryan Barrett: Replying to "How do we make data ..."
https://bsky.app/profile/bnewbold.net/post/3l2jhuf4jpm27 is the current best practice, Boris linked to it earlier
01:20:06 Nick Gerakines (he/him): 2 keys ... for now
01:20:11 Laurens Hof: Reacted to "Thanks everyone, got..." with โค๏ธ
01:20:17 Ryan Barrett: Reacted to "yes! my point was ve..." with ๐
01:20:23 Genco: (which is also controlled by the pds for 6 million users)
01:20:36 Paul Frazee: Replying to "2 keys ... for now"
pray we do not add another
01:20:57 Boris Mann: RDF solves this
01:21:09 will: Replying to "How do we make data ..."
I was thinking of a lexicon directory where you can view other people's lexicons etc
01:21:12 Paul Frazee: Replying to "yes! my point was ve..."
yes thatโs what the โrecovery keyโ is for, the idea is that you can store that on paper and pull it out when needed if your pds betrays you
01:21:29 Paul Frazee: Reacted to "RDF solves this" with ๐ฅ
01:21:31 Ryan Barrett: Replying to "RDF solves this"
bad boris! bad!
01:21:32 Paul Frazee: Reacted to "Thanks everyone, got..." with โค๏ธ
01:21:44 Ryan Barrett: Replying to "RDF solves this"
no treat for you
01:21:47 Boris Mann: Replying to "RDF solves this"
Donโt worry. I crushed a LOT of RDF dreams at DWebCamp
01:21:51 Genco: yes, what i was saying was that close to no one is holding their rotation keys, and there is close to no tooling around it
01:21:55 Ryan Barrett: Reacted to "Donโt worry. I crush..." with ๐
01:21:56 Boris Mann: Replying to "RDF solves this"
โIt hasnโt worked for 10 years, why should it start workingโ
01:22:19 will: we had this talk last night about a schema registry essentially
01:22:23 Nicolas Gimenez: I think probably interop will stem from actual use-cases between appviews
01:22:25 Paul Frazee: Replying to "yes! my point was ve..."
yeah fair. hopefully thatโs just a matter of time
01:22:28 Laurens Hof: Reacted to "mk ultra activation ..." with ๐
01:23:03 Ryan Barrett: Reacted to "yes, what i was sayi..." with ๐
01:23:27 Laurens Hof: The good news is that the atmosphere is already ahead of fed as this point
01:23:53 Laurens Hof: As the link-aggregator people donโt talk to each other at all
01:24:04 Nick Gerakines (he/him): +1 Smoke Signal's first major rev and reset was a lexicon update to bring it closer to {"text": "event description ..."}
01:24:08 Rudy Fraser: Or com.atproto itself
01:24:27 Damien Sedgwick: ATmosphere
01:24:33 Nicolas Gimenez: Just thank you for the amazing presentation!
01:24:53 Boris Mann: https://lu.ma/ag1uwvss
01:24:55 will: lovely space to be a dev
01:24:56 Bryan Newbold: strong analogy to software libraries for Lexicons: there is tension around "in std library or not", and when language ecosystems converge on matrue popular libs or not
01:25:12 Nick Gerakines (he/him): on the record fits with atproto
01:25:16 Ryan Barrett: Reacted to "lovely space to be a..." with ๐
01:25:18 Ryan Barrett: Reacted to "strong analogy to so..." with ๐
01:25:44 Ryan Barrett: Reacted to "on the record fits w..." with ๐
01:25:45 timothy: Nice to get on the AToll
01:25:47 dan: Reacted to "on the record fits w..." with ๐
01:25:52 Rudy Fraser: Reacted to "lovely space to be a..." with ๐
01:26:01 Boris Mann: Weโre saving the chat and will share it as well
01:26:16 Nicolas Gimenez: Reacted to "Weโre saving the ..." with ๐
01:26:23 Bryan Newbold: Reacted to "Weโre saving the ..." with ๐
01:27:17 Genco: ooh this is also how video works right
01:27:31 Tom: Super cool!
01:28:31 Ryan Barrett: YES
01:28:36 Nick Gerakines (he/him): Rich text over html scrubbing and sanitizing any day.
01:28:38 Ryan Barrett: markup ftw
01:29:42 Genco: lexicon resolution would be really cool!
01:29:52 Ryan Barrett: Reacted to "lexicon resolution w..." with ๐
01:31:39 Ryan Barrett: shades of XML etc
01:31:59 Nick Gerakines (he/him): Reacted to "shades of XML etc" with โ
01:32:04 Nick Gerakines (he/him): Reacted to "lexicon resolution..." with โ
01:32:57 Nicolas Gimenez: Reacted to "shades of XML etc" with โ
01:33:00 Paul Frazee: I was moving super fast there so I hope all that made sense
01:33:03 Nicolas Gimenez: Reacted to "lexicon resolution..." with ๐
01:33:09 timothy: Well done, Tom
01:33:11 timothy: Legend
01:33:15 Paul Frazee: awesome stuff tom
01:33:15 Nicolas Gimenez: ๐
01:33:16 Ryan Barrett: congrats Tom! Frontpage is awesome!
01:33:18 dan: thanks to boris for organizing too!
01:33:19 Ryan Barrett: and thank you Boris!!!
01:33:22 Tom: You you all so much, lovely group of folks!
01:33:23 Max Saal: Thanks so much all! My first time on an atprotocol call, interesting stuff
ATProtocol Dev is a blog, newsletter, and event series following the development of protocol apps building on top of ATProtocol, which powers the Bluesky micro-blogging network and an emerging set of open social apps.