Making a software trailer with Astra

I launched a fun little OSS library yesterday, TrustMe. I originally built it as part of another project (more details soon).

TrustMe brings a better interface for granting 3rd party application accessibility permissions. MacOS defaults force an obnoxious blocking window to the front of the stack, which then sometimes opens the Settings application but sometimes doesn't. It's inconsistent, and the broader permission UX has gotten worse, with macOS Sequoia introducing recurring prompts to re-authorize screen recording.

TrustMe instead asks users to drag and drop. We'll open the settings window for them, and provide a floating window where they can accept permissions by just dragging it to the approved list. There's no abrupt notification - and we can seamlessly detect when permissions are granted so users can move on.

Permission UX is hard to sell in a README, since people mostly notice it when it's bad. So I wanted a short trailer that shows the annoying version first. Here's how I (or perhaps more accurately my agent friends) filmed it in an hour.

Why Open Source

Before we get into the meat of the trailer - I want to spend a second addressing the obvious question of why spend the time/tokens to do this in the first place.

There's a vocal contingent that argues OSS has run its course. Unless you're an established large project¹, it's hard to get support from good maintainers, you're stuck dealing with a ton of vibe coded contribution spam, and your users are one prompt away from replacing you for good.

I still believe open source has a bright future. Especially so for opinionated design. LLMs will bend to any opinion that you give them, but are relatively inconsistent over time in how they structure projects and design features. Taste is still very hard to reward in an RL environment. TrustMe is an instance of very opinionated design, which I think is worthwhile. Whether users take it as-is and install it in their swift project, or just use it as inspiration to build something else - that's up to them. That's really the whole reason why the MIT license was invented in the first place!

This is notably not artisanal code that I hand rolled. The main value that I provided, as a human, was overseeing the architecture choices, the design, and the UX to get it to a place where the flow feels intuitive yet new. I'm also putting my name on the thing, which matters if you trust my own bar of quality. That's hard for an agent to fully replace.

For a brief moment OSS maintainers had star power granted to them by the community. It's possible those days are fading; as agents become the decision makers there will be fewer people passionate about one repo over another. But having things in the open and shared between companies has good network effects and herd protection.² So open source we will!

Stage Direction

My core message is to juxtapose the annoyance of permission notifications during app installations, with the serenity of a plugin that can manage it for you. I knew from the get-go the outline of what I wanted:

Start with a computer with obnoxious notifications
Each message should appear with pings and spooky music
Move to a scenic series of clips: dogs, flowers, or the ocean
Finally demo our TrustMe UX
Drag a new app to the permissions window, see the confirmation

The early part should get people stressed. The middle segues into calm. The final use of the UX rides the same wave of calm to the better UX.

Whatever though - it's a short trailer not a Christopher Nolan film. But if you got a chance to watch the final product, I'd say it actually executed pretty well on this initial theme! So mission accomplished against our desired storyboard.

Full AI End-to-End

I wanted to see what Astra could do here while controlling an arbitrary video generation model. I gave it credentialed access to Runway and let it cook for half an hour. It could spend as many credits on whatever video model it wanted - at least up to my plan's limits. The above was the result.

It's quite bad - but at least conceptually it's following my spec. My biggest complaints were on the rendering quality. The most lifelike scenes (like the dogs) are squarely in the uncanny valley. I made sure to put our desire for a fully realistic, cinematic rendering in the system prompt yet here we are.³

Plus: the computer logic was far off from what I wanted. The general animation of screen content worked alright but text sometimes gets fuzzy / hallucinates. More critically some of the UX patterns just aren't possible; for instance in the last frame, the TrustMe popover disappears into the settings menu like no mac app actually would. I'm trying to demo my package after all. It needs to look real.

Looking back on the thought traces, Astra constructed most of these videos by generating some keyframes with Nano Banana and letting the model tween them. I suspect I would have done something similar if I were prompting because it's the only way to actually crack some of the UI design. But still it was too far away from what feels real.⁴

Swift Visuals

Video generation isn't going to cut it, so let's try actually going to the studio and grabbing our video camera.

I wanted the depth of field of a real laptop, with real pixels being filmed. But that means that I couldn't really operate the laptop while I was filming it. Should I use some screen controls software to animate the cursor? Just replay a quicktime video? We also had to solve for the annoyance of the popups. How can we prompt macOS to pop up a tremendous amount of permission notifications for a single application?

I eventually went with an easier option. All of the computer footage that you see in the video was fully headless. The wallpaper was painted and the "applications" were implemented as separate Swift windows in one fullscreen app, down to the drag motion of going into the Settings dialogue. It imports the full TrustMe library to mirror the UI design from there as well.

The two demos were implemented by Opus. Both of these scripts lay things out pretty similarly:

The desktop is my real wallpaper (boring default yeah I know). We can scan for this at a low level with CGWindowListCopyWindowInfo and isolate the screen capture with the screencapture cli
That gets painted full-screen with a fake menu bar on top
Every demo window sits above .screenSaver level so it blocks all other apps and notifications

The popup storm⁵ starts with a single Accessibility prompt. Another one comes after, then the pacing quickens until they arrive every few hundredths of a second. There's some variation in the prompt types as well (camera, screen record, keychain, etc) just to add some variation. Placing popups randomly left some visual holes, so it used a coarse grid to track what's already covered, until the screen is completely buried.

The TrustMe demo is similarly synthetic. The System Settings window is just a re-implementation of the one that is actually on the OS. So we have finer grained control for what shows up in the list, and can auto-approve the fingerprint prompt. The drag visual is also a transparent overlay with the system cursor images.

So basically - everything is fake! Thank you movie magic.

All of this, down to the implementation choices and tuning of the animations, was based on a very high level prompt that I gave to Opus with some tweaks along the way:

I want to demo ../trustme. I want to do so in two scripts: a swift app that does
a full-page simulation of accessibility popups. should be styled in the same way
that the OS systems are, but unlike the system ones we should fully control them.
we should animate it so: it starts on an image of my desktop that I'm currently
using (screenshot it - no icons just the graphic), then lays out the popups. the
first one in the center, starts slow, then progressively more and more to cover
the screen. esc to escape from it. then another script that launches an animation
that just shows our trustme window and a simulated settings window, and shows
the drag to be added to the settings window.

The determinism of controlling the whole generation process meant that I could film, re-setup the shot from another angle, and click the same script to get the same exact output. I bet directors wish they could do the same with some of their acting talent.

When we were done moving around the camera we had about 15 different clips. At each distance I got at least two shots, one of the popup script and one of the trustme solution. I wasn't sure how I'd want to cut them together.

imported_footage

Astra as editor

I both needed to splice these visuals into a story, and needed a fitting soundtrack alongside the pings of notifications.

After some false-starts with using computer use to automate Final Cut, I checked if there was a programmatic way to edit. Turns out that letting Astra just do things in scripts was far more effective. The best edit was done with a build-time script, written with some combination of python, ffmpeg, and opencv.

The final model message thinks of the episode as eight shots:

Shot	Length
Continuous laptop push-in	9.81s
Notification close-up	6.01s
A woman shushing (Pexels)	1.27s
Ocean (Pexels)	2.50s
Daffodil bloom (Pexels, sped up 2.47×)	3.00s
Permission selection	3.20s
Granted state	4.40s
TrustMe closing card	3.20s

Since Astra is not a native video model, I suggested it work around this by rendering contact sheets along the way. Contact sheets are just a concatenation of a lot of different single image frames together, sampled at regular intervals. This gives it a good sense for the overall flow of the content without having to watch the video in the first place.

All of the audio is synthesized in numpy. The pings are a sine wave with a couple of quieter overtones and a fast decay. The shush and the ocean are random noise shaped with an FFT. The calm score plays piano-like notes over a sustained chord, like this:

def piano(freq, seconds=4.7):
    t = np.arange(round(seconds * SR)) / SR
    envelope = (
        (1 - np.exp(-110 * t))
        * np.exp(-t / 2.2)
        * np.clip((seconds - t) / 0.6, 0, 1)
    )
    tone = sum(
        amplitude * np.sin(2 * np.pi * freq * harmonic * t)
        * np.exp(-t * harmonic * 0.1)
        for harmonic, amplitude in [(1, 1), (2, .28), (3, .12), (4, .035)]
    )
    return tone * envelope

Taking the shush noise as one example: we had a separate stock video & audio that we needed to align. The exact place to position the audio was determined by these contact sheets and checking individual frames. The pings were synced in a similar way. Popup arrivals were detected by diffing frames of the laptop screen since the camera stayed in the same spot.⁶

This video was basically one-shotted, with the addition of a small edit I requested that swapped out the logo and got rid of a secondary camera cut. There was also a weird issue with the zoom-in of the first clip shaking very subtly. Astra was able to track down this shake in the zoom; zoompan rounds the crop to whole pixels, so it rewrote the zoom with subpixel cv2.warpAffine. Measured jitter dropped by almost 90%.

All of this was done in natural language, I didn't touch the editing timeline during the edit.

It's a super duper simple edit, but I'm still left impressed. I gave it raw assets and a general plan to implement, and it executed basically to my artistic vision. It even introduced the cut between the first notifications and zoom in shot that I had not planned yet worked better than what I had in mind. The particularly nice thing about this use-case (versus - say - production code) is you can really stop caring about how the sausage is made. As long as I end up with a render and a Final Cut timeline that gives the output I want, I can throw away all scripts in the middle.

Conclusion

Lesson learned. Video models still can't fake real UI. Writing the UI in Swift and just filming the laptop worked much better.

But that was half the fun! The combination of human-machine interaction made this whole side-quest way more rewarding than it would have been with AI generation alone. It also left me feeling more creative than I sometimes do using AI tools. It was really using them as a tool - a source of other creative ideas - instead of just delegating the process end to end.

Overall - there's no excuse now for your open source project to have some shitty branding. Make it cool! Make it fun! I think you'll feel a lot more fulfilled even if you're just doing it for yourself.

Or are able to ride the AI hype wave of the moment. ↩
Security by transparency (and enough people/agents working to hack it) will always outweigh security by obscurity. ↩
My guess is Astra didn't properly critique the original images generated as reference, combined with inherent limits to the video models. ↩
Our bar for realism on screen is pretty damn high because we spend so much time seeing OS patterns repeat themselves. ↩
This is what Opus called it but it just sounds spooky doesn't it. ↩
The reconstruction of what these scripts are doing is just what I'm doing for this blog (and assisted by Astra). ↩