Handling of filenames with umlauts and other special characters

[phorward]

This problem currently exists in ViUR.
A file named testupload- ö ä -(123).txt should be uploaded.

1. The file testupload- ö ä -(123).txt is selected or dropped.
   
2. file/getUploadURL is being called with this filename, and generates an upload URL to Google Cloud Storage with the cloud bucket and file path project-id.appspot.com/06eJqwTaFo8JO/source/testupload-%20%C3%B6%20%C3%A4%20-%28123%29.txt.
   
   This first sanitizes the filename to the above filename form (source).

        fileName = sanitizeFileName(fileName)
   
        targetKey = utils.generateRandomString()
        blob = bucket.blob("%s/source/%s" % (targetKey, fileName))
        uploadUrl = blob.create_resumable_upload_session(content_type=mimeType, size=size, timeout=60)

   Then, it creates a new file leaf skel with the filename pending in (source):

        fileSkel = self.addSkel("leaf")
        fileSkel["name"] = "pending"
        fileSkel["size"] = 0
        fileSkel["mimetype"] = "application/octetstream"
        fileSkel["dlkey"] = targetKey
        fileSkel["parentdir"] = None
        fileSkel["pendingparententry"] = db.keyHelper(node, self.addSkel("node").kindName) if node else None
        fileSkel["pending"] = True
        fileSkel["weak"] = True
        fileSkel["width"] = 0
        fileSkel["height"] = 0
        fileSkel.toDB()

3. The file is being uploaded to Google Cloud Storage using the provided upload URL.
4. Afterwards, file/add is being called with the key of the file leaf skel created by file/getUploadURL. This now reads the Google Cloud storage object blob and extracts the filename again from it (source):

            skel["name"] = utils.escapeString(blob.name.replace("%s/source/" % skel["dlkey"], ""))

   Resulting file entity in Google Cloud Datastore:
   
   Wow! Such a nicely resolved piece of code.
5. The file is uploaded. The filename is displayed "broken" everywhere it is being used. A simple string unescaping is not enough, as the filename is URL-encoded. Why? For security reasons?
   

In the end, this entire stuff is inconsistent and breaks the filename.

1. In step 2: Why not save the pending file leaf skeleton with the unquoted filename, like testupload- ö ä -(123).txt (pending)
2. In step 4: Why is skel["name"] being created from blob.name? Why is 06eJqwTaFo8JO/source/testupload-%20%C3%B6%20%C3%A4%20-%28123%29.txt turned into testupload-%20%C3%B6%20%C3%A4%20-%28123%29.txt and then the URL-escaped string is utils.escapeStringed again. Is this extra-secure? Why not just use the original filename provided before without the (pending)-postfix, and this, as normal for StringBone content, is required to be utils.escapeStringed.
3. General question: Why is ViUR required to save escaped data for any string content? Why is such data not escaped when rendered, so the HTML-renderer is the place where data is going to be escaped, so no code leaking or injection will be possible. In JSON or any other format, the escaping is irellevant and impractical.

[phorward]

#1050 has a new naming scheme which is allowed for filenames, so a paranoid escaping like here should not be required anymore. Filenames not following the naming scheme will be rejected.

[phorward]

Hopefully fixed by #1050, otherwise please reopen this issue.
Existing code and uploads are still provided with the escaped filename.
They should be re-uploaded. @xnopasaranx