Prior ffmpeg implementation provided insufficient precision for frame
estimation and was heavier than desired. Regexes are no longer used
Round frame_rate annotation to at most 2 digits
Previously, detailed response codes were used to indicate what went
wrong on a failed server request. However, this produces un catchable
error messages which are undesirable. Instead, these responses now use
204 to indicate that the request was successfully processed, but returned
no content.
About a year and a half ago, ComfyUI changed it's input validation to
allow for specifying which inputs should be validated, but removing the
undesired inputs would cause errors for users who had not updated
ComfyUI to recieve this update. As a result, input validation emthods
were modified to accept an additional kwargs input to allow
compatibility with both the new and old versions. Around the time
execution inversion landed, this was again changed so that a kwargs
input could be used for validation with dynamic inputs. From this point,
the kwargs argument also served to prevent input validation.
Supporting extremely old (>1 year) ComfyUI versions is not currently a
major priority, so these kwargs inputs can be removed to restore input
validation
Resolves#413
Query video isn't denounced and winds up being checked frequently. This
can produce heavy console spam when source information is frequently
updated. More cleanup will be done, but for now, an empty response is
returned instead of an error to prevent the console from getting
spammed.
Adds a new node to select the most recently modified file of a target
folder. This can be used to automate using the output of one execution
as the input for another. In order to allow frontend values to properly
update, this is implemented as a virtual node, skeleton node def is
included due to difficulty with setting up the node as fully virtual.
This will likely be revisited in the future.
Add an optional length argument to path trimming. The original 30
character length estimate seems to no longer be accurate which results
in text overlapping. A more thorough investigation and fix will come
later.
Update experimental designation on nodes.
Despite initial best efforts, the subprocess code for preview
transcoding causes frequent issues with blocking the server thread.
Instead, the asyncio methods for subprocesses are now used instead.
Generating accurate frame counts becomes fairly involved when other
widgets are modified that affect frame number. In an effort to
centralize this frame count logic, the loaded number of frames is now
calculated on the backed queryvideo call and this number is then
displayed on in the annotation.
Add try/catch loop to the queryvideo call. This failing is expected if
the ComfyUI server is restarting or down and this ensures exception
doesn't propagate and cause issues elsewhere
The frame_load_cap widget will now display the actual number of frames
that will be loaded if it differs from the selection
The queryvideo endpoint now wraps its information under a source item.
If information is later added to process other query args, those may can
now be added under a loaded item instead.
A queryvideo endpoint has been added to fetch information about a video
file including resolution, duration, and number of frames. This does not
currently process any query args, so number of frames will need to be
sacaled for target fps.
Add a option to do strict checking on the number of frames and raise an
exception if this is not met. If strict checking is not enabled for a
format, log an error instead. This is subject to change since runtime
exceptions are heavy handed even when opted into.
Modify the ffmpeg output parsing for fps to be safer.
When downloading previews with yt-dl, do not block
Functioning server code was in place to pull image_load_cap and
skip_first_images when processing previews in the Load Images nodes, but
these args where being redirected to function on the images after they
have been converted into a video. This was causing issues, so this
remapping has been disabled.
Resolves#368
Added a check to ensure a format input exists on load nodes before
trying to initialize it.
HTML source elements claim to provide a way to have different priorities
for videos served. I had hoped that this would provide a way to minimize
the chances that a users configuration would result in previews failing
to display, but from testing, this appears to be quite unreliable.
updating the src on source elements doesn't cause the browser to
recheck which source items are valid and and I noticed requests
persisting after elements have been removed.
Since this update set is already becoming larger than expected, I plan
to revisit this functionality at a later date.
Functionality to skip encoding in the video preview endpoint has been
reduced. This functionality created security concerns by exposing read
access to any file on the users system. Instead, it only displays files
that already can be handled by the native view endpoint and only
executes in the fringe case that a user install is incomplete and ffmpeg
is unavailable.
Further old code related to force_size has been removed. The viewvideo
endpoint will likely be changed in the future as well, but I'm more
cautious of other custom nodes that leverage this endpoint.
Use new format to define settings as part of the extension
Update calls to get setting value to no longer provide a default value,
This was once required, but is now deprecated and produces console spam.
Introduce a skip encode parameter which returns the file itself. This
allows the preview endpoint to be used to reference video files by full
path, without requiring re-encoding
Add a minimum width setting.
With the fix to make read non-blocking, the preview code would busy
loop. This made concurrent display of previews non-functional and had
poor performance.
To handle this, the read has been given a larger buffer (1MB) which is
unlikely to saturate, a miinimum loop duration is added of .1 second,
and the fallback sleep (which is unlikely to ever be called) is made
async as well.
The previous except cases were insufficient. For example, if the ComfyUI
server was killed while a preview is being produced, then a BrokenPipe
error message is displayed.
Sometimes hours of hairpulling end in a trivial solution.
Because a maximum size was not specified for the read operation, the
read operation was blocking until ffmpeg had finished producing the
entirety of its output. By specifying an arbitrary, (but fairly small)
size, video files now properly stream and the delay between a preview
being requested and being first display is both significantly reduced
and no longer scales with the length of the previewed video.
I had thought I had tested for this specifically and found that external
media players would properly stream previews, but I am regardless glad
to have things functioning as designed.
I've had underlying concern for a while now that universally using utf-8
isn't correct because it doesn't respect locale and this serves as
fairly strong confirmation. From a little bit of digging, checking seems
as simple as calling `locale.getencoding()`, but the documentation
claims that this is ANSI for windows (meaning changing it would affect
the majority of users). As a result, I've refactored out all the string
decoding to use a common variable and set it to display an escaped
version of any unconvertable characters, but am leaving the format as
utf-8 until I have further information.
See #324
VHS adds server endpoints to get a file listing for a directory and to
process video files.. These endpoints were not namespaced to VHS, but
should be.
As external sources (Even the core frontend) referrence the existing
endpoints, they are left in place, but can be deprecated later down the
line.
A couple of miscellaneous fixes after introduction of the load ffmpeg
nodes.
Refactor js code to be a bit more general to the inclusion of multiple
load nodes.
Add an experimental Load Image Path node.
Interest has been raised for adding a dedicated Load (single)
Image node to VHS. From my experience, ffmpeg itself has the most broad
support for conversion between image formats. Using Pillow would provide
limited benefit over the core Load Image node and the implementation
cost here is negligible as the node can use the existing Video Loading
infrastructure. My expectations are that ffmpeg will have better support
over opencv for loading singular images, but I think it best to push it
through as an experimental node. If it proves non-viable, it can always
be removed later. Additional functionality (Mask derivation?) can be
added later.
Load Video (Upload) and Load Video (Path) will now display files with
the MOV extension and will ignore case when checking if a file is in the
list of allowed extensions.
As a workaround to displaying transparent videos a to support correct
audio synchronization in advanced previews, a initial prepass is
performed to query information on the video file. This prepass
mistakenly used a hardcoded `ffmpeg` instead of the proper ffmpeg_path
Updated ProRes to allow transparent output when set to Profile 4. As
part of this change, pix_fmt is no longer explicitly set for ProRes, but
the expected format is automatically applied from for each profile.
Added rgba as an output pix_fmt for the nvenc formats. These have not
been tested and may not function.
Fix incorrect escape in Advanced Preview code.
Executable preference is
VHS_YTDL environment variable > yt-dlp > youtube-dl
Downloaded videos are cached in temp so they are not preserved between
reboots
Both Load Video (Path) and Load Audio (Path) are supported
On Load Video, Size scaling had a major rewrite to support different
downscale ratios, but logic was flawed. This has been fixed
Image Sequence previews were broken by adding a framerate arg since the
input is a sequence file and not the images themself. This could be
added later by adding the speed information to the sequence file, but
has been removed for now since there isn't any infrastructure to set the
frame rate on the Load Images node itself
Added an 8bit-png output format. Lots of people seem to prefer
outputting image sequences. When the extra quality isn't required,
having a separate 8bit format allows for a significant speedup and less
disk space used.
Frame rate is now passed so image_sequqnces play back at proper rate.
Context menu options on an image sequence (open preview, save) now
operate on the first frame instead of throwing an error.
There were still intermittent errors and console spam from a transcoded
advanced preview being interrupted before it could finish.Connection
errors are also caught now.
ffmpeg does not appear to handle transparency as expected for gif
creation. Transparent areas will show the previous frame, or a portion
of it. A deep dive into trying to find configuration options to fix this
did not result in solutions that functioned on my machine, but did
introduce me to the ffmpeg split filter.
Split allows for a single ffmpeg execution to generate a palette and
utilize the palette for creating the output gif without requiring a temp
file. However, the video data must be buffered in memory by ffmpeg.
Quick testing seems to show that ffmpeg uses 4 bytes per pixel
of buffered frame data.
The ramifications for memory usage are nuanced, but wind up preferable.
The increased memory per pixel of ffmpeg's buffering only results in
(approximated) 3% memory increase over the whole workflow, but remove
the need for a tempfile, a second subprocess call, and allow for
imperfect Meta Batch support which can only extend the total frame count
by 5-6x. It's a reduction in moving parts for underwhelming memory
characteristics in both the best and worst case. Additionally, it likely
actually addresses the issue the original requester saw in #160
The actual code for allowing pre-passes is left in for now and will
likely be re-evaluated for other uses or pruned at a later time.
Actually bump version.
All path stripping is now performed by a dedicated strip_path function.
First all whitespace is stripped, then a single double-quote is stripped
from the start and end of the path. Subsequent double quotes or white
space inside of the quotes is maintained.
Fixed a number of places paths received no stripping
- Load Video (Path) execution
- Path queries
- Advanced Previews
- Load Audio
See #224
Add a 16bit-png video_format which supports output transparency. Other
formats could be updated for similar capabilities, but are likely to
have issues with device support and unlikely to see widespread adoption.
Modifies the preview server code to properly parse and check for the
existence of sequence filenames.
After substantial rewriting, paths are declared as strings in the python
code, and converted to path widgets in the webui if they include an
option for vhs_path_extensions. resolves#104
VALIDATE_INPUT and IS_CHANGED for path nodes were rewritten and largely
consolidated in utils to accommodate validation of inputs not known before
execution.
Known issue: VALIDATE_INPUT is never called for load_audio which results
in error messages that aren't very helpful if a bad path is provided
An error is no longer immediately raised if ffmpeg is not found
Internal paths are now absolute for load images advanced previews
Instead of requiring VHS_UNSAFE_PATHS to disable checking paths on the
new endpoints, checking is disabled by default and VHS_STRICT_PATHS can
be used to re-enable it. This makes the added functionality opt out,
instead of opt in, as has been a primary goal, but the quality of life
has won me over, the difference in availability is minimal compared to
the functionality of the existent path nodes and I suspect setting would
cause unnecessary burden upon the average user.
Fix an improper check in the js code causing previews to break when
force_size is left on disabled.
Previously, force_size information was discarded when requesting
an advanced preview as the video was instead scaled to the size of the
node. Fixing this was deemed particularly difficult since the python
code does not have access to the actual video resolution before starting
the ffmpeg process.
This is solved with a more complex ffmpeg command that passes the
cropping logic to the ffmpeg subprocess.
In an edge case, scan dir would throw an erro when encoutering a broken
symlink. It has been fixed to ignore these broken symlinks
Fix an edge case where the video_formats folder is overridden by
ComfyUI-AnimateDiff-Evolved if VideoHelperSuite loads first. See #87
Further README additions
Basic support for loading urls has been added to Load Video.
Most of the work is made trivial by both ffmpeg and opencv automatically
fetching a url if it's provided instead of a path.
A more sophisticated fix for error passing from ffmpeg during preview
generation. If a request is aborted, the ffmpeg process is immediately
killed instead of being allowed to respond to stdin being closed and
waited on at the end of the with block
The toggle for saving metadata has been changed back to a boolean
instead of a combo. While I wished to provide for future compatibility,
the change broke existing workflows and there was no accommodating
implementation to load metadata saved under a different tag.
The log level for ffmpeg preview processes has been raised to fatal.
There's a good chance this will make debugging harder for users when
errors arise, but aborted requests were too common for the error
messages produced by them to be acceptable.
Format widgets for ints are now properly rounded.
Load Images had a typo prevent it's extension filter (of nothing) from
applying
Audio inputs and outputs now consistently use lowercase audio as a
description and have a format name of VHS_AUDIO. While other extensions
could share the audio type name, they would not pass functions.
Previously when using ffmpeg to load a video, care was needed to ensure
that the first frame was index 0, but the frame at 1s was the first to
appear after 1s. This should now be fixed.
The js code for load video nodes now passes the extension as part of the
format so that gifs and webps can be properly distinguished between when
deciding if an advanced preview should be loaded.
Fixed a mistake in serialization of custom height on load video nodes
Austin Mroz