• 0 Posts
  • 9 Comments
Joined 2 years ago
cake
Cake day: July 4th, 2023

help-circle



  • I’m not pretending to understand how homomorphic encryption works or how it fits into this system, but here’s something from the article.

    With some server optimization metadata and the help of Apple’s private nearest neighbor search (PNNS), the relevant Apple server shard receives a homomorphically-encrypted embedding from the device, and performs the aforementioned encrypted computations on that data to find a landmark match from a database and return the result to the client device without providing identifying information to Apple nor its OHTTP partner Cloudflare.

    There’s a more technical write up here. It appears the final match is happening on device, not on the server.

    The client decrypts the reply to its PNNS query, which may contain multiple candidate landmarks. A specialized, lightweight on-device reranking model then predicts the best candidate by using high-level multimodal feature descriptors, including visual similarity scores; locally stored geo-signals; popularity; and index coverage of landmarks (to debias candidate overweighting). When the model has identified the match, the photo’s local metadata is updated with the landmark label, and the user can easily find the photo when searching their device for the landmark’s name.


  • It’s not data harvesting if it works as claimed. The data is sent encrypted and not decrypted by the remote system performing the analysis.

    From the link:

    Put simply: You take a photo; your Mac or iThing locally outlines what it thinks is a landmark or place of interest in the snap; it homomorphically encrypts a representation of that portion of the image in a way that can be analyzed without being decrypted; it sends the encrypted data to a remote server to do that analysis, so that the landmark can be identified from a big database of places; and it receives the suggested location again in encrypted form that it alone can decipher.

    If it all works as claimed, and there are no side-channels or other leaks, Apple can’t see what’s in your photos, neither the image data nor the looked-up label.


  • Here’s a lake effect snowfall map for the US. Indiana does get some, but for reference, I’m talking about the red (96-150 in / 244-381 cm) and deep red areas of upstate NY. It’s a whole different kind of snow when that stuff starts falling hard.

    Scenes that look like this are peak winter IMO. It’s an alien landscape. Everything is so clean and quiet.


  • Don’t ever move to a lake effect snow area… I miss those nights when multiple feet fell on us in upstate NY and the world was completely transformed. Outside sounds like an anechoic chamber from all the fat snowflakes absorbing all the sound; the crunch underfoot being the only thing you hear. Shoveling out the driveway to go rip e-brakes in the snow covered parking lots. Winter without snow is a half assed experience.

    Ice can fuck right off though.