The definition of partner sharing

[ttyridal]

Hi All,

Do the Immich community have, or is able to reach some kind of common understanding of what partner sharing is/contains?

It seems a lot of people think the current state is not going far enough:
See/share partners persons/faces: #6339 #5089 #5457
See/share partners photos on map: #6339
See/share partners album (automatically shared): #7026
"Full access" (shared faces/tagging/albums): #7029 #6335 (?)
Implementing tags would very likely spark requests to be shared too
(i've obviously missed several references above)

I'm keen on tackling this (also to scratch my my own itch) Probably staring with partner's faces, map and memory becoming "first class citizens" in the shared-to's UI.

My question is really, where does partner sharing end? My own use-case would typically allow for both parties having full control over shared resources (ie both can add / remove faces/assets to a person). But it might be taking it a step too far? (I don't plan on including this in the first round)

Then, starting with faces sharing, there are immediately a couple of challenges.

• A person might occur both in the primary users photos and the in-shared photos. (ie admin sharing person-A, user1 also has some photos of person A in her own library). Current plan is to leave user1 with two person-A objects, but ideally they would be the same. That would require some handling if the partner share is removed. I think the most user friendly behaviour would be to fork the person-A object if it is unshared?
• In the above case, when merging two person objects, the shared object should be preferred ?

It's probably worth while to consider #4500 (share subsets of a collection between different users) in the same context. I think it would probably require a new access concept (something like group ownership?) If the setup is not too complex, it might be sufficient with a separate user owning the different sets/libraries - But very cumbersome if only the owner can tag, manage faces etc.

[ttyridal]

From PR comment by @martabal (#7045 (comment))

I agree but that's a technical challenge as soon as you have multiple partners : Let's say you have users A, B and C. If users A-B and A-C are partners, B and C must not see each other's faces / people.
Overall, it's a security and performance challenge

It could kind of work i think. Assets will be protected in all cases: The backend looks up assets linked to a person that the user has direct or partner.sharedTo access to.
Haven't checked what's controlling access to the cover photo itself.

Now, for the person objects it's getting more interesting. If partner shares are one-way (user A is basically a "resource account") it will work, I think.
Given the strategy suggested above: user A has a person X. User B also has some photos of X and merges the two, ending up with person X being owned by user A and referencing assets owned by A and B. User C will see the same person X, but only the assets owned by user A - as expected.

If the partner share on the other hand is two-way, we have no obvious strategy for automatically deciding the ownership of person X. It may work as above or disappear for user C.

[MikeBishop]

Piling on here. I was exploring Immich for use with my family -- initially my wife, but presumably eventually also further shared with our kids as they get older and have devices. I'm finding that the work I've done on identifying matching people doesn't carry over to her account, despite us having pictures of basically the same people. It's frustrating that son-taken-by-dad and son-taken-by-mom are effectively different people; clearly they're all just pictures of our son. (I almost think that people and face matching should be global state, not user state, since identity of the person in the photo is generally an objective reality.)

Similarly, in trying to clean up the import, I quickly discovered that while the UI will let me select a new date for her photos, that doesn't actually seem to go anywhere. I can understand that being a permissions thing, but as I do the bulk of the photo cleanup for both of us, it means logging in as her.

I wonder if there's an element of request-consent here; I'm allowed to propose changes to her pictures (match with this person, etc.), but the next time she logs in, she'll see a list of changes I made and be asked to approve them before they're "committed." Or, alternatively, more granular settings up front in sharing -- not only do you select a partner, you decide whether they can see albums, people, etc. and whether they can edit the things they're allowed to see.

[ttyridal]

I can like your track-changes (propose-review-commit) idea. It would be particularly (?) relevant for deletes. At the moment we're using an album as the "suggesting to delete" box.

We're getting into some rather specific use cases though. "full-access/equivalent to owner" would probably cover 99.9% of the needs.

Regarding the globalness of a person. I think a challenge will be that it's easy to disturb the algorithm. When matching a new face, it finds the closest existing one and, (if within a threshold) assigns that face's person to the new face. If you can get enough faces where the distance is just within the limit, you will be making a "catch all" person.

Again it falls a bit down to who the audience of Immich is. Within a household/family (trusted environment, hopefully :)) that's probably acceptable, but I have the impression the team is targeting a more general userbase.

[aero31aero]

On a technical level, I feel that there should be a table that links pairs of matched people across users. So, entries like:

{
  entry_a: {user: dad, person: kid-taken-by-dad},
  entry_b: {user: mom, person: kid-taken-by-mom}
}

would comprise this table, say, faces-pairs, and any time a search is performed, if mom searches for kid-taken-by-mom, look up face-pairs table and then any photos of kid-taken-by-dad owned by dad but visible to mom should be shown in the search results.

Internally, the face recog algo doesn't need to know anything specific about these faces; just merge them in the data layer.

This also solves some cases like I want to label my grandma as 'Dadi' and my dad wants to label her 'Mummy', so we can have our own names for people.

[frejaya]

Very interested in this! At the moment partner sharing isn't very functional for my needs. We have thousands of historical photos and want to easily navigate them, so if we both want the full functionality of explore and search, we have to upload our photos to a shared admin account. Meanwhile we also want to back up our phones automatically to our personal accounts and have some control or privacy over which ones make their way into partner sharing (archive function seems to meet this need). There doesn't seem to be a realistic way at the moment without logging out/in whenever we want to search our shared library vs. backup our phones.

[Yetangitu]

Here's another use case for (possibly optionally) sharing 'people' when such are present in shared photos: sharing family photos with my mother who unfortunately has Parkinsons disease and as such - due to her trembling hands - tends to often press buttons and links and options and other things-on-screen without meaning to do so. This leads to all sorts of problems like her accidentally deleting things which she does not want deleted.

Giving my mother access to a shared account would quickly mess up the configuration in many interesting ways because of the same problem. I can give her read-only access to a shared library which does not suffer from these problems but, alas, the main attraction - that of being easily able to find photos of specific people - is not part of partner sharing. This has kept me from giving her any access to my installation for now, instead relying on Jellyfin and Nextcloud Memories for the same tasks. Jellyfin is read-only but only offers such organisation as I manually put in place, Nextcloud Memories comes closer to what Immich offers but is decidedly slower and not as effective in its facial recognition no matter which of the related 'apps' is used. I'd like to give her access to my current Immich instance and will do so as soon as shared 'people' is implemented (with a read-only share option...).

[popy2k14]

+1 for people/faces sharing. My wife also wants to search for faces i have tagged/named (ex. our son).
Would be nice this feature will come in an future version.

[Yetangitu]

(#7045 (comment)): I think it would be better for us to wait and implement it the proper way

What would "the proper way" look like? In the end it will always come down to a given user starting a scan which produces a number of clusters. Those clusters are added to labelled groups - "Joe" and "Alice" and "Torbjörn". Those groups are what are called "Persons". The scanning and clustering is handled by the machine, grouping is left to the user so that is the first step which can be clearly related to a user and where there may be differences in opinion on whether cluster 1 belongs in the "Joe" group or maybe in "Bob". Now give a situation where both user1 and user2 have access to a set of images encompassing the mentioned clusters, either through a share or because they configured the same external source in their libraries. Who gets to "own" the clusters given that these are machine-generated? Maybe they should be treated like any other metadata related to the image (location etc.)? If so, should anyone who has access to an image also have access to the cluster metadata for that image?

Groups are created by users but it would be a waste of effort for all users to need to group and label clusters. That does not mean groups can be easily shared just like clusters are since:

• user1 may think cluster-0001 belongs in group "Joe" while user2 things it belongs in group "Bob"
• user1 may call this person "老子" while user2 - who does not read Chinese - calls him "Lao Tzu"
• user1 may have attached a label to some group which user2 might not be all too happy about ("My annoying uncle" vs "Bob")
• user1 may prefer thumbnail_12345 to represent "Alice" while user2 prefers thumbnail_abcde

From this I gather the problem can be broken in two parts which each need to be solved:

1. clustering metadata is machine-generated and relates to individual images, not to individual users. This means it can be shared when needed without privacy repercussions, especially since every user could have this data generated on demand. This clustering metadata is stored in asset_faces in the database where it has two FK relations: assets(id)->assetId and person(id)->personId.
2. clusters are grouped and labelled by users into 'persons'. Those labels - names or descriptions - as well as the related thumbnail represent specific user choices which should only be shared explicitly by the user and which may not always be relevant to other users. If user1 shares this information with user2 the latter may want to edit it to fit his preferences. Such editing should not affect user1 unless the edit represents a correction or addition which is relevant for both users. Groups are recorded in the person table with two FK relations: users(id)->ownerId and asset_face(id)->faceAssetId

As it currently stands the schema does not allow separation between clustering and grouping metadata since asset_faces refers to person (which contains user-specific information which needs to be shared explicitly, not implicitly) while person refers to users (which means only a single user can 'own' the cluster).

Here's a few options on how this could be handled:

1. clustering metadata is seen as image-related metadata and available to any user who has access to the image, whether through sharing or by adding the directory which contains that image to their library. If user1 shares the library with user2 he gets the option to share related persons (i.e. groups with names and thumbnails, what are called 'persons' now). Such information is shared read-only with a copy-on-write option, i.e. as soon as user2 edits either name, birthday or thumbnail for a given group a new group is created in which this information is stored. This new group replaces the shared group for user2 but it does not affect user1. User2 can choose to share back this edit to user1 who can opt to replace his version with the edited one. If he does so this does not affect user2 so if user1 reverts the edit user2 will still have access to his own edited version. For persons to which user2 has not made edits any edit made by user1 also affects user2.
2. Clustering metadata is shared as described above. If user1 shares the library with user2 he gets the option to share related persons either read-only or editable. For persons shared read-only the same copy-on-write option as described above is used when user2 edits an existing person. If user1 chose to share persons read-write any edit done by user2 affects user1 as well. This is effectively the same situation as when using a 'shared family account', i.e. any user with access can edit persons.
3. [wasteful but for completeness' sake] Clustering metadata is shared as described above. If user1 shares the library with user2 he gets the option to share related persons. If he does so user2 gets his own copies of all person metadata. If user1 edits any person metadata after sharing this does not affect user2.

WRT shared person-related metadata, what should happen when user1 undoes the share with user2 while user2 had edited and/or created his own persons for that library? Should these be retained - possibly for a limited amount of time - just in case the share is re-established?

[Yetangitu]

I described that situation in the first reason why groups (not clusters) can not easily be shared:

• user1 may think cluster-0001 belongs in group "Joe" while user2 things it belongs in group "Bob"

I make a distinction between the clusters as produced by the model and the groups to which these are assigned by the user. It is there that the decision is made whether a cluster gets labelled "Joe" or "Bob" and with that there that users can have differing opinions on which name to attach to a group. A single group can contain one or more clusters, a cluster can never belong to more than one group. Groups are currently referred to as "Persons", clusters are what the model initially presents in the "Persons" interface. Sometimes two or more clusters end up showing the same individual upon which the users groups them together (after having confirmed that they are actually the same person), hence the distinction between clusters and groups.

I do not foresee a situation wherein clusters are anything but machine-generated because I do not see how one can be made otherwise given the way these are defined: the machine finds faces in images which are clustered together based on a certain criterium of 'sameness' or 'proximity'. Users are not presented with individually recognised faces but clusters of faces which, according to those criteria, are deemed to represent the same person.

So the clusters are machine-generated and not in any way editable by users, hence the suggestion of having these be part of the image metadata and handled as such. What the user does with this information is where user privacy and user choice comes into play. This is what I named 'grouping'. Many groups will contain only a single cluster but some will encompass many more - I have groups with more than 10 clusters on my local installation with ~200.000 images and ~600 labelled groups and a starting cluster count of around 1600.

[ttyridal]

What the user does with this information is where user privacy and user choice comes into play. This is what I named 'grouping'

But isn't this then exactly what we want to share and (potentially) co-edit? after that process, the original clusters are not very interesting anymore.
Each new face is compared to all (accessible) faces/assets and the closest group is selected.

[Yetangitu]

Yes, it is, and that is where the privacy boundary goes as well as far as I can see. Before this it is just the machine creating a bunch of embedded vectors which are clustered programmatically according to some criterium, after that it is user-chosen input which sometimes should be shared, others times not. It is also here where users may want to make different decisions based on language preference ("老子" vs "Lao Tzu"), personal preferences ("My annoying uncle" vs "Bob"), differences in opinion ("Joe" vs "Bob") and different thumbnail choices. Which leaves the question on how this should work when shared persons are edited. Should user2 (the one accessing a shared library) simply be allowed to edit user1's (the library owner) persons, should a copy-on-write scheme be used so that user2's edits do not affect user1?
Part of what I'm philosophising about is mostly relevant for external libraries which can be accessed by multiple users - whether through shares or because they chose to add them to their own libraries. Still, even with a shared personal library the question on how to handle edits by the 'receiving' users. Am I overcomplicating matters by thinking different people may want to use different labels for groups? Maybe this is not what Immich aims for - at least not at this stage - and with that all users who have write access to a share should be able to edit at will with the edit affecting the share owner as well?

[alextran1502]

What would "the proper way" look like?

In my vision, a proper way of sharing facial collection is when you choose a partner, and with the permission of the partner, the facial of that partner's account becomes yours, and a mechanism to recluster everything and redistribute among your and the partner's account, keeping the facial data with the same name.

If you withdraw to be the other person's partner, you will get your pool back and a job to recluster your pool.

That is the coarse vision of the implementation

[ttyridal]

@alextran1502: great! upfront, sorry for being a bit slow 🙈

So, when userA enables partner share to userB (with all necessary permissions) userB can:

1. see the union of userA and userB person objects in various places in the UI (basically as implemented by feat(server, web): partner share persons/faces #7045)
2. Edit userA's person objects as if they where her own (change names etc..)
3. What about changing face-person associations on assets belonging to userA ?
4. Associate faces from her own assets to person objects originating from userA
5. Merge her own person objects with person objects originating from userA (basically a batch of 4)
6. Consider person objects originating from userA as candidate for match when adding new assets
7. Queue a re-cluster job that will consider person objects originating from userA as candidate for match (?)
8. Automatically merge person objects ?? ("recluster everything")

If share broken:

9. All person objects originating from userA (only those with associated faces from assets owned by userB?) shall be cloned ?

or are all faces (now) associated with person objects originating from userA set to unmatched?

Your're doing a distinction between partner share without faces, with faces read-only and with faces "as-my-own"?
(so if "read-only" only (1) applies?)

Thoughts about the cases mentioned above in this thread, ie A shares to B shares to C etc?

[TwoD]

My hopes for sharing includes being able to share a whole library, with associated persons, albums and everything else, with several members of my family - but without sharing my personal library/libraries with them.

I've begun categorizing lots of old inherited photos from an imported library and I got a fair bit along before realizing I can not currently share just this library with a group of people. The library files themselves are read-only, but I would like the others to be able to contribute new albums for this "shared space" (and tags when that comes along).
The library is fairly large and will contain many albums when fully categorized/tagged, so I do not want each user to have to duplicate the massive effort doing it all from scratch.

This is part of building a general digital archive + backup solution for which I'll be the main caretaker. I have done an initial rough categorization and assigned the majority of faces already (which do greatly overlap with my personal libraries), but the others would like to be able to contribute once the library is shared with them. If possible, we would all be able to selectively include photos and people from outside the shared library. Essentially I want them to be able to say "person A in my private libraries = person B in the shared library" and "these photos from my private library should be included in some of the shared albums", for things like events they also had photos from.

Had I realized the current limitation of partner sharing sooner I may have elected to import the to-be-shared library to a new user instead of my own user. Then we could all add that user as a partner. It however appears that may not give the type of "selective overlap" I need. If A & B & C share library X, while B & C share library Y, I do not want that to mean A & C also share Y. But, if a face exists in all of their libraries I do want that face to be "shared", or at least "linked" so that no matter who searches for that person (by whichever name they assigned to them), also see all the photos shared with just them.

My expectation for what happens if any two people stop sharing libraries/albums is that they retain at least the albums and/or persons/faces that had been associated both with a shared library and their own library, and any changes thereafter are completely independent.

When initially sharing a library (and thus faces/persons/albums) with another user, I would expect that user to have that library pop up under the "Shared" section, along with a question about how to integrate it with their own libraries (or other libraries shared to them, if allowed). I'd like Immich to be able to scan for similar faces across shared libraries and automatically suggest a "soft merge / linking", so that I can actually decide myself if Person A from my user truly matches Person B from their user. If so, I'd still see them as "Person A" after the merge, while the other user(s) would essentially just see "Person B" having a bunch of new photos associated with them - of course also limited to only the photos from albums or libraries shared between the two users.

I realize this next part will to a great degree overlap with the other discussions about concepts like tags, and "rule based" albums, so I wasn't exactly sure where to put it, but hope you don't mind I put it here.

Conceptually, I can see how this could be implemented as "Hierarchical, linkable Tags with ownership". (Ignore the actual current implementation and performance considerations for now, I do not know enough about Immich's code to actually say how big of refactor this implementation would entail.)

Basically, a library/album/person/face/location/whatever is all internally represented by a type of Tag entity.
Tags have an optional parent tag for defining hierarchical taxonomies, and access control list for defining ownership(s)/access level.
In addition, two tags can also be bound together by a Link entity. A link always contains two users (which may actually be the same user) and two tags. When Immich applies a Tag to a photo, any associated Links are fetched and compared to the ACL of the applied tag. If the ACL says the other user in the Link has at least "read" access to the first tag, the other linked tag is also applied.
Note that the other tag's ACL may not have an entry for the first user, so applying both tags could be a one-way operation - allowing one-way sharing.

All the photos in "Album Foo" for user 1 would be tagged "album:foo". Creating an album implicitly grants you full ownership.
"Album Bar (album:bar)" was created by user 2, and then shared as read-only with user 1 - adding user 1 to the ACL for "album:bar" as read-only. (The ability to re-share tags may also be a separate permission.)
User 2 then under "Album Bar(album:bar)" adds "Album Baz(album:bar:baz)", which inherits its ACL from "album:bar", also granting user 1 the same access as they had to Bar - except here they also get granted write-access.

User 1 has their own libraries, albums or other tags completely independently, except for this one photo they'd like to contribute with into Bar, which means it internally gets tagged with at least (album:bar:baz).
Let's User 2 has already tagged "Person Q (person:q)" in this photo. User 1 won't see that face matched unless there's already a link to Person Q into their own collection of people. If there is a link to between user 2's Person Q and user 1's Person P, those "person:p" tags from User 1 are also applied to that photo.
If there isn't already a link, Immich would still run a face match on behalf of user 1 for this new photo and suggest a match for "person:p" as usual.
If the users at a later point do set up a link between Persons Q and P, the image has already been tagged correctly. Any new images uploaded to - or later tagged with - an album or library both users 1 & 2 have access to would be scanned for matches as usual, but if a match for a linked tag is found, all other linked tags are also applied.

If user 2 (owning the album) tries to delete this photo (still also having tags owned by user 1), all they can really do in practice is to remove that the tag(s) user 2 owns from it, effectively un-sharing it with themselves. It won't actually get deleted unless user 1 also removes all their own tags from it. If it's an un-shared image without any faces or albums, this effectively just removes the "library:#" tag from it, and any un-tagged photos are deleted. If the library is read-only, the library tag's ACL prevents its own removal.

Either of the users could also opt to un-link Persons P & Q, effectively preventing Immich from auto-assigning both those tags when a match for either is detected in any library or album shared between them. At this point one may want Immich to ask if it should also de-associate any existing images with the previously linked tags. Perhaps the server owner could also decide what to do beforehand, or limit the options available to the performing user in situations where bulk associating/applying/removing [linked] tags would show up.

I think if sharing is done in this way, or something similar, it may be flexible enough to more or less let users define how sharing is supposed to work for them. Immich needs only to make very few "business logic" assumptions about associations between users and can "just" look at any Tag's ACL and matching Links, to determine what other tags to apply.
Maybe it could eventually look at some ruleset, or even fire some event that custom code could react to, if the default behavior is not enough.

[rkirmeier]

I want to second the idea of having a family library run alongside a personal one, saving my user base from having to remember a shared account along with a personal account, not to mention having to consistently switch between them. Having read all of thee above, I don't think there is anything I might add besides pushing back on the idea that this is a niche application of this product.

The role of family archivist is not a small portion of the prospective immich userbase (labeled above as the single enthusiast with little family ties). I will give that there are plenty of single people in this generation. However, it is safe to assume a majority had parent(s). Most of those had their own parent(s), and every single one of them has a box of dusty photos, waiting to be digitized and properly archived for a hope of passing them down. Even from outside this generic family setup, a found family will want the same control alongside the same privacy.

There are other markets they could target.
-Example: The designer/creator market (doesn't seem likely with the focus on facial recognition and the lack of a dynamic tag system). these would not see any use in a feature like this unless thee worked with a team and also used the software for personal/side business.

• just the privacy advocate who wants to dump google/subscriptions. (not mutually exclusive with family archivist)

family archivist may not be the majority user demographic, but i do think it is probably the biggest subset of potential users, especially in a multi account setup.

i do think the facial recognition is a very small part of the feature being proposed in the initial paragraph. I would hate that being the feature holding up developing what the process for a shared library/album/file set would look like.

[DeltaTango69]

I imagine the whole implementation to be quite complicated. A shared account would actually be enough for me, so all rights are also shared and you only have one database. It's just that I do all the work with assigning people etc.. My wife just wants to see the results and doesn't want to create or change any data herself. She would then express her wishes to me and I would implement them. So there is no need for the program to be able to do all this with any suggestion lists and confirmations etc.. It's all far too complicated and nobody can keep track of it!

The only thing that bothers me is that all new pictures from the different smartphones end up in the same directory. The simplest thing would be to create subfolders here that are assigned to the different devices. Then we can use Immich with a shared user. If someone doesn't want to share their personal pictures with their partner, they can always create a separate user for them. Keep it simple!

[popy2k14]

I imagine the whole implementation to be quite complicated. A shared account would actually be enough for me, so all rights are also shared and you only have one database. It's just that I do all the work with assigning people etc.. My wife just wants to see the results and doesn't want to create or change any data herself. She would then express her wishes to me and I would implement them. So there is no need for the program to be able to do all this with any suggestion lists and confirmations etc.. It's all far too complicated and nobody can keep track of it!

That's exactly what i thought partner sharing should be.

[ihlstbrown]

+1 for face sharing/editing as well. I have over 280k of assets and having someone help me identify faces would be great.

How about exposing the folder structure as well to the partner? I have external libraries that are organized by folder already, and with so many assets it's often easier to look at the photos in a smaller subset (like a folder). I know I could use filters for some of this, but folders would be handy.

[rkirmeier]

I think a folder structure is pretty antithetical to how the front end of most of these photo software's work. Not that there isn't value in the folder structure. I just don't see it getting a frontend implementation. What you could do is upload each folder set into an album (or multiple albums if you have a lot of layers to your folder system). You could also share a path to the file directory for easier (if less feature rich) access. I think you can even set up immich to mimic whatever folder logic you have been following. These are just some suggestions in leu of frontend folder support.