Event Aggregator

[andy]

Hi there

There’s no good one-stop shop to find all the various events happening in a given city, so I’m trying to build one!

I’ve got a fairly basic implementation up here.

At the moment, it just runs a daily cron that scrapes events from a small set of local Vancouver publications/venue websites then pushes them to a bluesky collection, which the site then fetches and uses to populate the calendar. It supports basic commenting and RSVP functionality via bluesky auth.

It’s very rudimentary at the moment and the UI is atrocious, but I figured I should just get it up there and make improvements later. The mobile experience isn’t great and I didn’t even add a loading state during the login process yet. There’s also a dark mode, but I put it in the authenticated user dropdown for some reason lol. I’ll tidy all that up soon!

Note: there’s a small gap from the last couple weeks because I didn’t realize my github credits ran out, so the cron wasn’t running and I haven’t backfilled the missing events yet

Here’s a little more info

High-level overview of current features

• Automated Scraping:

  • Event scraping CLI that can be executed on a cron(such as a GitHub Actions workflow).

• Data Quality Filtering:

  • De-dupes scraped events
  • Filters malformed events before publishing

• AT Proto Integration:

  • Publish events to your Bluesky repo using the community calendar lexicon.
  • Publicly read events from any DID repo without authentication.
  • Automatic deduplication prevents duplicate events.
  • Conform event data to the AT Proto Smoke Signal format
  • Comment posting and moderation on individual events via Bluesky OAuth
  • RSVPs and moderation on individual events via Bluesky OAuth
  • Community-specific moderator whitelists(via delegation)

• Calendar UI:

  • Renders events using a basic calendar with source filtering and event details(first of many primitives in a series of Smoke Signal components).

• Local Cache:

  • Store events in data/events.json for fast unauthenticated reads. (Can interface with any external data store)
    • This can be used as a fallback if the bluesky collection isn’t available for some reason, but that’s only ever happened if I’ve pushed a breaking change without realizing it

In-flight features

• UI framework integration

• Manual event submission

• Moderator cohorts to approve events manually submitted by the community

• No-code scraping interface(also subscription-based intelligent scraper)

• Better mobile layout

Future features(short-medium term)

Enhanced Scraping

• Add More Event Sources:

  • More hardcoded event sources
  • Community-submitted events
  • Dynamic scraping interface where a user can simply submit a URL to scrape

• Improve Data Quality:

  • Better date parsing for edge cases
  • Location standardization and geocoding
  • Event categorization (music, theater, sports, etc.)
  • Price information extraction

User Experience

• Calendar Improvements

  • Event search and filtering by keywords
  • Category-based color coding
  • Mobile-responsive design improvements
  • Export to personal calendars (iCal, Google Calendar)
  • Interactive map with event locations
  • Ticket purchasing

• Performance Optimization

  • Event pagination for large datasets
  • Caching strategies for faster load times
  • Image optimization and lazy loading

API Enhancements

• REST API Expansion
  • Event search endpoints with filters
  • Venue-specific event listings
  • Event statistics and analytics

[boris]

Welcome @andy! This time it was me taking forever to respond.

The big thing I wanted to share with you was Slices https://slices.network – this can aggregate events lexicon and give you API end points so you can focus on just a front end client.

And would still like to figure out what hosting a Smoke Signal node would look like for Vancouver – Z-Space and DWebYVR and whatever I do with ATProto Vancouver would all be folks that would jam with this since I’m connected to them, and we could recruit more.

(and of course you definitely want your dedicated music stuff – you’ll have the bulk of events)

@knowtheory is talking to people in Oakland / Bay Area about some events stuff too.

[tompscanlan]

I’m also interested in the space. Reached out via bsky to you, but I’m here too for a public conversation.

I’ve had been aggregating local events from newspapers into platform.openmeet.net, but plan on allowing users to submit urls to parse/crawl for events.

Anyway, sounds like a few of us doing similar things, and maybe ripe for some collaboration. The work I’ve done on event ingestion hasn’t been made public yet, but happy to share with folks that are interested in a scrapy and chat gpt powered event web crawler.

[erlend.sh]

@tompscanlan

Anyway, sounds like a few of us doing similar things, and maybe ripe for some collaboration.

I was just about to send a nudge to both Open Meet and Smoke Signal on bsky to say “maybe these projects could collab more?”, so I’m very happy to see you here Tom!

[tompscanlan]

Glad to be here, and hear from you.

Did you have any specific collaboration in mind? I’m very open to collaboration, but I’m also not great at making or keeping connections. I’ll try to turn it on for a bit and check in with folks.

I was hoping @andy would let me know more about his project, as I saw him starting to feed Vancouver events. I noticed the feed is not running right now, or maybe I’ve got something down. How’s it going Andy?

[boris]

I dragged Andy in here and he’s got some stuff up and running, but we’re both just occasional on this.

Like I mentioned in the other thread, maybe a community events call is a good idea.

I’m traveling and I briefly talked to @ngerakines about it, maybe early December?

[ngerakines]

Yeah! I saw a bunch of events come in. I think there are a few small issues with the records preventing them from being processed by Smoke Signal and showing up. I think the main thing is that members of the “locations“ array need to have “$type” fields set.