Einer von Vielen

open source face recognition usable with Darktable (the backend is Deepface)

Fri, 24 May 2024 21:57:07 +0200 zuletzt bearbeitet: Fri, 24 May 2024 22:04:43 +0200

See the code. It's basically a Java frontend for deepface that uses exiftool to write metadata into pictures in a standardized way so that it is readable by darktable and other programs that comply to this standard.

face_recognition darktable deepface

Cloud file content not complete if opened in webbrowser

Einer von Vielen

Wed, 24 Aug 2022 15:21:23 +0200 via streams

@mike (mike@unfediverse.com)

What am I doing wrong?

An addon stores a json using $file->put(json_encode($faces_replaced));
Example

{
"id":{"0":"AsGPgFifqwBRXSr","1":"KERRkttLsZbwkrk"},
"file":{"0":"handy\/2022-05-25_21-22-27_Abendessen in Laag - GM1917.jpg","1":"handy\/2022-05-30_09-27-49_Ivan im Caf\u00e9 in Monaco - GM1917.jpg"},
"position":{"0":[53,37,29,54],"1":[14,74,25,55]},
"face_nr":{"0":1,"1":1},"name":{"0":"ivanu","1":""},"name_recognized":{"0":"","1":""},
"time_named":{"0":"2022-08-24T12:42:30+00:00","1":""},
"exif_date":{"0":"2022-05-25T21:22:27+02:00","1":"2022-05-30T09:27:49+02:00"}
}

What is working
The content of the file is correct if opened in the file system directly, example var/www/mywebsite/store/oj/98216e6f-f392-4af9-ac71-671e4688c03e/fc641786-f49f-47ea-8f41-cc4e2cd7520d
(If php reads the file using Code\Storage\File the file content is correct, as well as read by the python backend that reads the file from the file system directly.)

What is not working
If the file is opened in the webbrowser, example http://localhost/cloud/oj/handy/faces.json, the browser shows "SyntaxError: JSON.parse: unterminated string literal at line 1 column 404 of the JSON data". Indeed, if the file is downloaded the file content is

{
"id":{"0":"AsGPgFifqwBRXSr","1":"KERRkttLsZbwkrk"},
"file":{"0":"handy\/2022-05-25_21-22-27_Abendessen in Laag - GM1917.jpg","1":"handy\/2022-05-30_09-27-49_Ivan im Caf\u00e9 in Monaco - GM1917.jpg"},
"position":{"0":[53,37,29,54],"1":[14,74,25,55]},
"face_nr":{"0":1,"1":1},
"name":{"0":"ivanu","1":""},
"name_recognized":{"0":"","1":""},
"time_named":{"0":"2022-08-24T12:42:30+00:00","1":""},
"exif_date":{"

If repeated it gives the same result.

This is a problem because if the browser tries to load the file via http://localhost/cloud/oj/handy/faces.json this will fail.

I keep searching for the cause...

addons

mehr anzeigen 4 Kommentare

Einer von Vielen

Wed, 24 Aug 2022 15:39:23 +0200 via streams

Please wait!

May be I found something.

Einer von Vielen

Wed, 24 Aug 2022 23:14:13 +0200 via streams

Did not had the time to figure it out completly but there is some automatic type conversion (objects/arrays to json forth and back) going on and it seems that pandas (Python) does handle json differently from PHP.

Einer von Vielen

Thu, 25 Aug 2022 14:17:58 +0200 via streams

This is still a mystery to me. If opened in the file system (text editor and python) the content is fine and complete. If loaded by the web browser the file is cut and therefor incomplete and a malformed json.

For now I find another solution.

post in admin page of addon does not work since commit 30140589c

Einer von Vielen

Mon, 22 Aug 2022 17:48:31 +0200 zuletzt bearbeitet: Wed, 24 Aug 2022 15:24:07 +0200 via streams

@mike (mike@unfediverse.com)

I pulled the dev today.

The admin page of my addon shows but the post does not work anymore (submit button).

I went back to previous commits and found that commit 3623e8a01 works and the next one 30140589c does not work.

A difference between the too is line 27 in Addon.php

$func($a);

versus

$func();

Is this a bug or a feature?

addons

mehr anzeigen 6 Kommentare

mike

Mon, 22 Aug 2022 22:21:24 +0200 via y

Just woke up. I'll sort it out as soon as some caffeine makes it to the brain.

mike

Mon, 22 Aug 2022 22:55:22 +0200 via y

OK. Should be good again. Thanks for the report. That was a change my IDE recommended and I thought it was dubious, but went along with it.

These days I have to rely on the IDE to catch some things for me; because sometimes I simply can't remember why some line of code that's 5-10 years old was written a certain way.

The force_queue setting is actually correct, although I agree the code looks wrong.

Einer von Vielen

Wed, 24 Aug 2022 12:57:12 +0200 via streams

tested. ok

Thanks

How to do get 100% width in the browser for an addon?

Einer von Vielen

Sun, 14 Aug 2022 17:06:48 +0200 zuletzt bearbeitet: Sun, 14 Aug 2022 17:07:17 +0200 via streams

#addons @Zap (zap@z.macgirvin.com) @zotlabs.org | Hubzilla Development (hubzilla-development@zotlabs.org)

Context
An addon wants to use the whole width in the webbrowser

Current solution
In PHP the addon uses replace_macros(Theme::get_template('xy.tpl', 'addon/xy'), array(....
In JavaScript the addon uses $("aside").remove();

Question
Is there a better way to do it?
(In Streams I found different Templates under Code/Render but are not sure how to use it correctly. I will have to do the same in Hubzilla.)

addons

mehr anzeigen 6 Kommentare

mike

Mon, 15 Aug 2022 02:24:58 +0200 via y

I answered this with the wrong identity so it might not have gotten through:

[template]full[/template]

should achieve what you're after. There are several ways of loading a Comanche layout for an addon. You can use the load_pdl hook (event listener) and set the 'layout' property on the callback array.

function myaddon_load_pdl(&$arr) {
   if ($arr['module'] === 'myaddon') {
       $arr['layout'] = '[template]full[/template]';
  }
}

Alternatively, if your addon is called 'xyz' and defines a controller (route) for 'xyz', we'll automatically look for mod_xyz.pdl in your addon directory if your addon controller responds at https://z.macgirvin.com/xyz.

Einer von Vielen

Mon, 15 Aug 2022 14:01:20 +0200 via streams

Alternatively, if your addon is called 'xyz' and defines a controller (route) for 'xyz', we'll automatically look for mod_xyz.pdl in your addon directory if your addon controller responds at https://z.macgirvin.com/xyz.

Worked

(This software is made to be extended by addons! Thanks)

Einer von Vielen

Mon, 15 Aug 2022 14:05:12 +0200 via streams

I answered this with the wrong identity so it might not have gotten through

correct

Ist there something like "touch"?

Einer von Vielen

Sun, 14 Aug 2022 16:51:24 +0200 via streams

#addons @Zap (zap@z.macgirvin.com)

Context

An addon calls a python script as backend. The backend changes the content of some files under webserver-root/store/nick.

Problem

The file attributes in the database (subsequently in /cloud/ ) are not updated

What I tried?

I looked at the class Code/Storage/File.php

Question 1)

I woud go for...

check wether the python script has finished (this has been accomplished for some other reasons)
check compare the modified time of the file under storage/nick/ with the "edited" timestamp in the database
update "edited" and "filesize" if needed

Is there any better approach?

Question 2)

Might it be usefull to implement a function "touch" in Code/Storage/File.php?

addons

mehr anzeigen 12 Kommentare

mike

Mon, 15 Aug 2022 02:19:16 +0200 via y

I'll have to give you the long story. The web interface only supports creating a new file (the name will be auto-generated) and will not or should not over-write an existing file. It "could" support updates but we felt is was best to not over-write existing files without some kind of warning.

The WebDAV interface supports updating a file and will re-calculate the size and re-generates thumbnails as needed.

So updating the database structures manually as you're doing now is probably the best way if you aren't using WebDAV. You could in theory also use a WebDAV client to store the file (you can see examples of such a client in python in util/nsh).

There is a configuration option which presumably allows you to update in place from the web interface, but I haven't used it and don't know for sure if it works.

You are welcome to try it and see if it works for you. Visit pconfig/system/overwrite_dup_files and set it to 1. Just be aware that this changes the default behaviour so it will always over-write your existing files unless you set it back afterward. And as mentioned it might not work at all. I wrote the code years ago but didn't test it because this is the first time anybody has asked for that ability.

Einer von Vielen

Mon, 15 Aug 2022 11:31:12 +0200 zuletzt bearbeitet: Mon, 15 Aug 2022 11:55:33 +0200 via streams

@mike (mike@unfediverse.com) Many thanks!

I think I was not clear enough what the addon is doing and what is needed.

After sleeping about my question I think a new function "touch" in Code/Storage/File.php would be the most elegant solution.

Why? The PHP code of the addon is the ONLY place to create or delete cloud files. It checks if a csv or pkl file is present in directories that contain images (jpg, png). If not it creates one.

Background
The data about where a face is positioned in an image will go into csv and the face representations (you have to compare to match faces) into the pkl file. This way the cpu and time consuming task of face recognition can be done either on the server or remotely via webDAV what is a big advantage to "unburden" the server. (Example: my notebook connects via webdav to the server. The script has no acces to the cloud files on the server and can do the heavy lifting of detecting faces and creating face representation, later on to match the presentation using so called distance metrics.)The addon will work with or without python running on the server. Without python running on the server the addon is still able to read the data (face position in an image) an show it in the webbrowser. If the users sets a name for a face the addon writes it into the csv. Holding data along with an image in the same directory is a disadvantage for the addon because storing the data in the database of Streams would be much easier to handle. On the other side a script can do the hard work remotly via webdav (what is the primary goal) and it is very easy for other addons/scripts/applications to use the data. This all is implemented and working. I give it more intense testing at the moment.

Back to the topic....
The Python backend will NEVER create or delete files. It just looks if a csv or pkl (binary for storing python objects) is present. If yes it reads the file content. At some point content is written back (to existing files only).

The python script will get the relativ path to webroot/store/nick with a query like this

query = ("SELECT display_path, os_path "
                 "FROM `attach` "
                 "WHERE `uid` = %s AND "
                 "`folder` = %s AND "
                 "`is_photo` = '1' AND "
                 "( `filetype` = 'image/jpeg' OR `filetype` = 'image/png')")

and use os_path to find the file in the filesystem of the webserver.

The filesize and timestamp change if content is written by python.

The "missing part"...
The database is not updated - and the file is not synchronized to clones.

Looking into Code/Storage/File.php the most elegant way would be to let this class check for an updated filesize and timestamp there and synchronize it there if any change happened.

The addon will iterate through the directories anyway to check the presence of a csv/pkl file. If it finds an existing one it could call a function touch in Code/Storage/File.php.

Of course the addon can implement it. This is ok but has the disadvantage of beeing not reusable and might not work over time. It seems more reliable to do this in a function touch in Code/Storage/File.php.

What do you think?

If ok I could come up with a proposal (code) for Code/Storage/File.php. (To be honest I trust you much more than me if it comes to code quality.)

Einer von Vielen

Mon, 15 Aug 2022 11:54:58 +0200 via streams

I think I was not clear enough what the addon is doing and what is needed.

After sleeping about my question I think a new function "touch" in Code/Storage/File.php would be the most elegant solution.

Why? The PHP code of the addon is the ONLY place to create or delete cloud files. It checks if a csv or pkl file is present in directories that contain images (jpg, png). If not it creates one.

Background
The name of a face and the data about where a face is positioned in an image will go into csv and the face representations into the pkl file. (The representations are used to match faces.) This way the cpu and time consuming task of face recognition can be done either on the server or remotely via webDAV what is a big advantage to "unburden" the server. (Example: my notebook connects via webdav to the server. A script on my notebook has now acces to the cloud files on the server and can do the heavy lifting of detecting faces and creating face representation. Later it match the face representations using distance metrics.) The addon will work with or without python running on the server. Without python running on the server the addon is still able to read the data (name of a face and its position in an image) and to show it in the webbrowser. If the users sets a name for a face the addon writes it into the csv.

Holding data in files is a disadvantage for the addon because storing the data in the database of Streams would be much easier to handle. On the other side storing the data together with an image in the same directory is an advantage because a script can do the hard work remotly connected via webdav (what is the primary goal) and it is very easy for other addons/scripts/applications to use the data. This all is implemented and working. I give it more intense testing at the moment.

Back to the topic....
The Python backend will NEVER create or delete files. It just looks if a csv or pkl (binary for storing python objects) is present. If yes it reads the file content. At some point content is written back (to existing files only).

The python script will get the relativ path to webroot/store/nick with a query like this

query = ("SELECT display_path, os_path "
                 "FROM `attach` "
                 "WHERE `uid` = %s AND "
                 "`folder` = %s AND "
                 "`is_photo` = '1' AND "
                 "( `filetype` = 'image/jpeg' OR `filetype` = 'image/png')")

and use os_path to find the file in the filesystem of the webserver.

The "missing part"...
The filesize and timestamp of a file change if content is written by python.
BUT the database is not updated - and the file is not synchronized to clones.

Looking into Code/Storage/File.php the most elegant way would be to let this class check for an updated filesize and timestamp. If any change happened it can update the database and synchronize the file to clones.

The addon will iterate through the directories anyway to check the presence of csv and pkl files. If it finds existing ones it could call a function touch in Code/Storage/File.php.

Of course the addon can implement it. This is ok but has the disadvantage of beeing not reusable and might not work over time. It seems more reliable to do this in a function touch in Code/Storage/File.php.

What do you think?

If ok I could come up with a proposal (code) for Code/Storage/File.php. (To be honest I trust you much more than me if it comes to code quality.)

Alexandre Hannud Abdo

Mon, 15 Aug 2022 13:23:44 +0200 via hubzilla

FWIW, you might want to reconsider using CSV and, specially, pickle here. For CSV you can easily use JSON instead (and adding a version number field to the stored object would be nice), it should make it much easier for other add-ons, or improvements to your add-on, to interoperate. And pickle is not a safe data exchange nor long-term storage format. Because you can run arbitrary code when loading a pickle, your use case looks like a security problem. It's also dependent on the version of Python. You should probably serialize/deserialize your objects as JSON (or another safe format) instead of pickling them. In which case you could even use the same file for both data if it makes sense.

Einer von Vielen

Mon, 15 Aug 2022 13:54:06 +0200 via streams

@Alexandre Hannud Abdo (aleabdo@hubzilla.com.br) Good points. Thanks for your thoughts! Nothing is release yet. So the best moment to change something would be NOW. Thinking....

Einer von Vielen

Mon, 15 Aug 2022 21:27:03 +0200 via streams

@Alexandre Hannud Abdo (aleabdo@hubzilla.com.br) Ok, I changed the file formats from csv to json and pkl to parquet in the python scripts. Now a lot to change in PHP and JavaScript

But this is just a matter of spare time.

How to "touch" a cloud file?

Einer von Vielen

Sun, 14 Aug 2022 16:46:05 +0200 via streams

#addons

Context

An addon calls a python script as backend. The backend changes the content of some files under webserver-root/store/nick.

Problem

The file attributes in the database (subsequently in /cloud/ ) are not updated

What I tried?

I looked at the class Code/Storage/File.php

Question 1)

I woud go for...

check wether the python script has finished (this has been accomplished for some other reasons)
check compare the modified time of the file under storage/nick/ with the "edited" timestamp in the database
update "edited" and "filesize" if needed

Is there any better approach?

Question 2)

Might it be usefull to implement a function "touch" in Code/Storage/File.php?

addons