●● IRC: #boycottnovell @ Techrights IRC Network: Sunday, January 09, 2022 ●● ● Jan 09 [01:06] *liberty_box has quit (Ping timeout: 2m30s) [01:12] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [01:18] schestowitz I have just tested the two-file script combo [01:18] schestowitz it worked great the first time around! [01:18] schestowitz I was surprised [01:18] schestowitz now, it still gave me this: [01:18] schestowitz stat: cannot stat 'test-video.webm': No such file or directory [01:18] schestowitz and no date in the resultant file [01:18] schestowitz the script is in git master [01:37] *liberty_box has quit (Ping timeout: 2m30s) [01:40] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell ● Jan 09 [02:00] *psydroid4 has quit (Ping timeout: 2m30s) [02:38] *liberty_box has quit (Ping timeout: 2m30s) [02:42] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [02:52] *SomeH4x0r has quit (Ping timeout: 2m30s) [02:56] *SomeH4x0r (~someh4xx@hngsz54a2nzc6.irc) has joined #boycottnovell ● Jan 09 [04:13] *liberty_box has quit (Ping timeout: 2m30s) ● Jan 09 [06:49] *DaemonFC has quit (Quit: Leaving) ● Jan 09 [07:00] *u-amarsh04 (~amarsh04@joseon-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell [07:00] *u-amarsh04 (~amarsh04@6pxtd49npuduw.irc) has joined #boycottnovell [07:52] Techrights-sec looking at the ffmpeg converson process and whether it can launch rsync, [07:52] Techrights-sec the size of the new file is tracked as it grows: [07:52] Techrights-sec ffmpeg -y -progress - -nostats -i ... [07:52] Techrights-sec That can be piped into awk with no buffers and processed, but in the case [07:52] Techrights-sec of transcoding the source file will be a very different size than the destination [07:52] Techrights-sec file. And in the case of only adding metadata there is no need to transcode [07:52] Techrights-sec so the copy option can be used instead. [07:52] Techrights-sec ffmpeg -y -progress - -nostats -i old.webm -c copy -metadata title="something" n [07:52] Techrights-sec ew.webm [07:52] Techrights-sec That should just take a few seconds. [07:52] Techrights-sec So would inserting the following at line 222 speed things up [07:53] Techrights-sec -c copy \ [07:53] Techrights-sec as in [07:53] Techrights-sec $ git diff preview.sh [07:53] Techrights-sec diff --git a/Desktop-Utils/preview.sh b/Desktop-Utils/preview.sh [07:53] Techrights-sec index 9ca7e53..98b1c28 100644 [07:53] Techrights-sec --- a/Desktop-Utils/preview.sh [07:53] Techrights-sec +++ b/Desktop-Utils/preview.sh [07:53] Techrights-sec @@ -219,6 +219,7 @@ echo ' =================================================' [07:53] Techrights-sec set -e [07:53] Techrights-sec nice ffmpeg \ [07:53] Techrights-sec + -c copy \ [07:53] Techrights-sec -f concat \ [07:53] Techrights-sec -i ./recipe.txt \ [07:53] Techrights-sec -metadata title="$VIDEO_NAME" \ ● Jan 09 [08:29] schestowitz-TR it would be nice to speed it up [08:29] schestowitz-TR but woulkd copy work when combining 3 files? [08:29] schestowitz-TR testing... [08:32] schestowitz [mov,mp4,m4a,3gp,3g2,mj2 @ 0x5644e4342700] Auto-inserting h264_mp4toannexb bitstream filter [08:32] schestowitz Unknown decoder 'copy' [08:43] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [08:44] schestowitz-TR I mfound the bug!! [08:47] Techrights-sec It might work when combining three files, but I haven't tried it that way. [08:47] schestowitz-TR they are also of different dimension [08:47] schestowitz-TR the commonality: audio rates, ratio, format [08:49] Techrights-sec Then reecoding must happen and that will be slow. [08:49] schestowitz-TR it is VERY slow indeed, but last year I researcher this and it looked like there was no other way, it's not a simple conversion like the first pass [08:55] schestowitz-TR see git [08:56] Techrights-sec Maybe it would be best to standardize the dimensions and so on for all the [08:56] Techrights-sec included files in advance. Then reencoding would not need to happen to merge [08:56] Techrights-sec them. [08:57] schestowitz-TR From what I vaguely recall, any time you merge files with different palettes and all, depending on format, the encoding stream needs to be redone [08:59] Techrights-sec ack ● Jan 09 [09:01] *liberty_box has quit (Ping timeout: 2m30s) [09:07] Techrights-sec Just checked, if the appended and prepended files can be made the same specs [09:07] Techrights-sec as the main video, then -c copy works well [09:07] schestowitz-TR the hard part is video dimension, not aspect ratio, as that used to vary a lot [09:12] Techrights-sec Hard to say, I have no samples which are not 1920x1080 [09:12] schestowitz-TR if we do tons of tidying up with those scripts maybe we can have a generalised framework for self-hosting videos with simple manuals. But it would take tons of work. [09:16] schestowitz-TR the videos I make are not always the same dims [09:16] Techrights-sec The videos listed in outro.list ought to be normalized to the same specs [09:16] Techrights-sec as the video you record. That way the processing is done just the once [09:16] Techrights-sec and then when you concatenate the files, ffmpeg can do it easily. Same for [09:16] Techrights-sec intro.list. [09:16] Techrights-sec Ah, why not? [09:17] schestowitz-TR depends which view it starts with, screen, face, external webcam (different aspect ratio) [09:18] Techrights-sec It'd save a lot of time, CPU, electicity, disk wear and tear, etc [09:28] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [09:31] Techrights-sec The size of the files can be estimated but not calculated until the processing [09:31] Techrights-sec actually is finished. Similar for counting frames, ffprobe has to go through [09:31] Techrights-sec the files linearly to get the frame count. But if the file sizes are used as an [09:31] Techrights-sec approximation for progress, then the size field from ffmpeg -progress - -nostats [09:31] Techrights-sec can be extracted using an AWK or Perl one liner and used to trigger an upload, [09:31] Techrights-sec if SSH keys are used. [09:31] Techrights-sec ack [09:33] schestowitz-TR see latest push [09:33] schestowitz-TR one other option I had in mind was running a timer for upload [09:33] Techrights-sec Hmm. How do make sure that ffmpeg has enough of a head start? [09:33] Techrights-sec You could use 'at' for upload. Or just let it go automatically when [09:33] Techrights-sec ffmpeg is done and then launch a popup notification when it is complete. [09:33] Techrights-sec But that's the current [09:34] schestowitz-TR it is not at all evident from the code, but atm what happens is, [09:34] schestowitz-TR I occasionalkly check the output and then drag and drop from dolphin [09:34] schestowitz-TR when the timing seems roughly right [09:35] Techrights-sec rsync or sftp could handle that automatically so you don't need to waste effort [09:35] Techrights-sec and can turn your attention to other things until the upload-complete notificatio [09:35] Techrights-sec n [09:36] schestowitz-TR the goal is to serialise things a little. it can save as much as an hour sometimes [09:41] schestowitz-TR ok, dumb question [09:41] schestowitz-TR can I mrate-limit an upload? [09:41] schestowitz-TR like, with scp or similar? [09:42] Techrights-sec yes with rsync or sftp, --bwlimit= in rsync and -l in sftp [09:43] schestowitz-TR that would still leave leave the challenge of correctly estimating the speed of upload, based on speed of size generation and its growth [09:43] schestowitz-TR *file generation [09:45] Techrights-sec ffmpeg -y -progress - -nostats -i old^Cebm -metadata title="so [09:45] Techrights-sec mething" new.webm | awk '{print $1,$6} END { print "111103897"}' RS='progress=co [09:45] Techrights-sec ntinue' OFS="\t" [09:48] schestowitz-TR trying this now on a real file [09:49] schestowitz z [09:49] schestowitz nice ffmpeg -y -progress - -nostats \ [09:49] schestowitz -f concat \ [09:49] schestowitz -i ./recipe.txt \ [09:49] schestowitz -metadata title="$VIDEO_NAME" \ [09:49] schestowitz -metadata creator="Techrights" \ [09:49] schestowitz -metadata \ [09:49] schestowitz license="Creative Commons Attribution-No Derivative Works 4.0" \ [09:49] schestowitz -metadata author="Roy Schestowitz" \ [09:49] schestowitz -metadata date="$(stat --printf='%y' $VIDEO_FILE.orig.webm \ [09:49] schestowitz | cut -d ' ' -f1)" $VIDEO_FILE.webm | awk '{print $1,$6} END { print "111103897"}' RS='progress=continue' OFS="\t" [09:49] Techrights-sec that just shows it's possible. working on something more practical... ● Jan 09 [10:02] Techrights-sec no need to run that one to completion, the AWK part adds nothing. It just shows [10:02] Techrights-sec that the running file size can be captured. [10:03] schestowitz-TR ok, gif parallelism works ok now [10:03] schestowitz-TR I suppose being able to trigger a function at 50% progress would help [10:16] Techrights-sec [10:16] Techrights-sec yes. testing atm [10:16] Techrights-sec unfortunately I chose a too long segment to test on, off-by-factor-of-one error [10:16] Techrights-sec ffmpeg -y -progress - -nostats -i x.webm \ [10:16] Techrights-sec -metadata title="someting" file.webm 2>/dev/null \ [10:16] Techrights-sec | awk -v f=$(stat -c "%s" file.webm) ' [10:16] Techrights-sec BEGIN { [10:16] Techrights-sec cutoff=int(f/10); [10:16] Techrights-sec print "Start uploading at ",cutoff; [10:16] Techrights-sec } [10:16] Techrights-sec !skip { [10:16] Techrights-sec split($6,size,"="); [10:16] Techrights-sec print size[2]; [10:16] Techrights-sec if(size[2]>=cutoff) { [10:16] Techrights-sec skip++; [10:17] Techrights-sec system("upload-script.sh &"); [10:17] Techrights-sec } [10:17] Techrights-sec }' RS='progress=continue' OFS="\t" [10:17] Techrights-sec the command subsitution part needs to return an integer which contains the [10:17] Techrights-sec approximate size in bytes of the combined files. [10:17] Techrights-sec Works with GNU Awk 5.1.0, untested in others [10:17] schestowitz-TR is this ready to integrate and test, or still in progress? [10:18] Techrights-sec Adjust the f/10 to f/2 or whatever fraction desired but it should work ok. [10:23] Techrights-sec The file name is hardcoded twice and ffmpeg concat and other options are missing, [10:23] Techrights-sec but it does trigger the script when the fraction is reached. So it will take [10:23] Techrights-sec a little to integrate it on your end, but hopefully not much. [10:23] Techrights-sec Are all the intros and outros more or less the same size? Or do they need to be [10:23] Techrights-sec measured each time? [10:23] schestowitz-TR they are 3 to 22 seconds, but compared to main "body" that's a fraction [10:24] Techrights-sec Ok but an integer needs to get fed to AWK [10:36] *liberty_box has quit (Ping timeout: 2m30s) [10:43] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [10:51] Techrights-sec filesize=$( [10:51] Techrights-sec awk ' [10:51] Techrights-sec /^file/ { [10:51] Techrights-sec cmd="stat -c \"%s\" " $2; [10:51] Techrights-sec cmd | getline s; [10:51] Techrights-sec close(cmd); [10:51] Techrights-sec size=size+s [10:51] Techrights-sec } [10:51] Techrights-sec END { [10:51] Techrights-sec print size [10:51] Techrights-sec }' ./recipe.txt [10:51] Techrights-sec ); [10:51] Techrights-sec ffmpeg -y -progress - -nostats -i x.webm \ [10:51] Techrights-sec -metadata title="someting" file.webm 2>/dev/null \ [10:51] Techrights-sec | awk -v f=$filesize ' [10:51] Techrights-sec BEGIN { ... [10:52] schestowitz-TR thanksd!! [10:56] schestowitz roy@vonick:~$ filesize=$( [10:56] schestowitz awk ' [10:56] schestowitz /^file/ { [10:56] schestowitz cmd="stat -c \"%s\" " $2; [10:56] schestowitz cmd | getline s; [10:56] schestowitz close(cmd); [10:56] schestowitz size=size+s [10:56] schestowitz } [10:56] schestowitz END { [10:56] schestowitz print size [10:56] schestowitz }' ./recipe.txt [10:56] schestowitz ); [10:56] schestowitz roy@vonick:~$ echo $filesize [10:56] schestowitz 2615970 ● Jan 09 [11:04] Techrights-sec np [11:04] schestowitz-TR I will push to git before testing [11:05] schestowitz-TR though I do wonder how rsync behaves when it handles an open and growing file, so I might need to add a new function [11:07] schestowitz-TR ok, pushed, not tested yet [11:14] *psydroid4 (~psydroid@cqggrmwgu7gji.irc) has joined #boycottnovell [11:28] Techrights-sec ack [11:33] schestowitz-TR testing now... [11:33] schestowitz-TR no error, I efresh server end to see progress as it went alon g [11:33] schestowitz-TR will test with a bigger file [11:33] schestowitz-TR will record somsething [11:34] Techrights-sec good [11:51] *wallacer has quit (Ping timeout: 2m30s) [11:51] *liberty_box has quit (Ping timeout: 2m30s) [11:51] *wallacer (~quassel@6bsu33ajs4zs4.irc) has joined #boycottnovell [11:53] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell ● Jan 09 [12:13] *DaemonFC (~daemonfc@tnxugezmmi87e.irc) has joined #boycottnovell [12:20] Techrights-sec good [12:20] Techrights-sec Current regulations regarding product liability seem to focus around goods sold [12:20] Techrights-sec and explicitly exclude services. An increasing amouunt of software is tied [12:20] Techrights-sec or run on remote servers, putting them into a hybrid category. As these lean tow [12:20] Techrights-sec wards becoming services (e.g. MSOffice) how much of that is being done as a [12:20] Techrights-sec dodge from product liability regulations? Softare is covered, technically, but [12:20] Techrights-sec ignored so far. Products, thus softwatre, are covered in particularly when they [12:20] Techrights-sec are used as advertised. Therefore when M$ victims use M$ products as adverstized [12:20] Techrights-sec and still get harmed, M$ is technically liable, even if the laws have not [12:20] Techrights-sec yet been enforced that way. [12:20] Techrights-sec See https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52018SC0157&from=EN [12:22] schestowitz sh: 1: uploadvideo: not found [12:22] schestowitz I think a variable scope issue [12:22] schestowitz when called from system() [12:22] schestowitz what's the simplest fix? [12:24] Techrights-sec See also: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:52021PC0206&from=EN [12:24] -TechrightsBN/#boycottnovell-eur-lex.europa.eu | NO TITLE [12:28] *liberty_box has quit (Ping timeout: 2m30s) [12:33] Techrights-sec sounds like a matter of the right path, try putting [12:33] Techrights-sec set -xv [12:33] Techrights-sec before the lines in question and then [12:33] Techrights-sec set +xv [12:33] Techrights-sec after them to show what thr sccript is looking for and the evaluate that in [12:33] Techrights-sec the context of the working directory to see if maybe the script should [12:34] Techrights-sec use paths diferently (e.g. set $PAtH at the top or else use absolute paths) [12:36] schestowitz-TR ok, see git [12:38] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [12:47] *DaemonFC has quit (Quit: Leaving) [12:51] Techrights-sec ack [12:51] Techrights-sec It's problem of quoting. Rather than fiddling with escaping the double [12:51] Techrights-sec quotes toedeal with a shell variable inside AWK, it would be easier to pass [12:51] Techrights-sec the shell variable into AWK using another -v, just a sec. [12:51] Techrights-sec yes, adding .... ok in git, but untested [12:51] schestowitz-TR can you push that in? [12:51] schestowitz-TR next i will do a pluton video [12:55] schestowitz-TR thanks, will record, then test [12:56] Techrights-sec ack ● Jan 09 [13:13] *liberty_box has quit (Ping timeout: 2m30s) [13:19] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [13:23] Techrights-sec back in a bit [13:56] schestowitz-TR I got the same error message as before the changes [13:57] *liberty_box has quit (Ping timeout: 2m30s) [13:59] *u-amarsh04 has quit (Connection closed) [13:59] *u-amarsh04 has quit (connection closed) ● Jan 09 [14:14] Techrights-sec back in a bit [14:14] Techrights-sec Does the script "uploadvideo" exist, if so is it in the $PATH? [14:14] schestowitz-TR it's inside the sourced file, which contains the function [14:19] Techrights-sec Ah, the system() function must not have the same material. I'll have to think [14:19] Techrights-sec a bit about how to get a function into that. [14:25] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [14:27] Techrights-sec Ok. AWK's system() uses sh instead of your interactive shell. I see a way [14:27] Techrights-sec to get it to use a function, but it must call the function file once again. [14:27] Techrights-sec Working... [14:31] *Despatche (~desp@u3xy9z2ifjzci.irc) has joined #boycottnovell [14:37] Techrights-sec ok a little complicated but try what's in Git [14:37] schestowitz-TR thanks! [14:40] Techrights-sec np [14:40] schestowitz-TR i will do a kristall video to test the changes with [14:42] Techrights-sec ack ● Jan 09 [15:12] *liberty_box has quit (Ping timeout: 2m30s) [15:15] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [15:50] schestowitz-TR the latest version (git, identical) seems to have worked, but it's still in progr [15:50] schestowitz-TR ess, so no leaping to celebration just yet [15:51] Techrights-sec ack [15:51] Techrights-sec if it gets too much more complicated, then perl would be the next step. [15:51] Techrights-sec Length is not so much a problem as complexity. [15:51] schestowitz-TR it is harder for me to cope with perl [15:52] Techrights-sec Ok, so incentive to keep [15:52] Techrights-sec the scripts simpler :) [15:56] *liberty_box has quit (Ping timeout: 2m30s) ● Jan 09 [16:05] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [16:32] *liberty_box has quit (Ping timeout: 2m30s) [16:33] *tech_exorcist (~tech_exorcist@x36jhwfgrvdfk.irc) has joined #boycottnovell [16:33] *tech_exorcist has quit (connection closed) [16:38] *tech_exorcist (~tech_exorcist@qiahibkh2iwka.irc) has joined #boycottnovell ● Jan 09 [17:08] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [17:19] Techrights-sec back in a few hours [17:44] *tgraswe (~bodg@tqt9fr4m7w6w4.irc) has joined #boycottnovell [17:47] *tgraswe has quit (connection closed) ● Jan 09 [18:06] *tgraswe (~bodg@vpjntwzzzc8xi.irc) has joined #boycottnovell [18:07] *tgraswe has quit (connection closed) [18:07] *tgraswe (~bodg@v8nadz8adup3w.irc) has joined #boycottnovell [18:36] *DaemonFC (~daemonfc@c6f2nzqiwd2bc.irc) has joined #boycottnovell [18:42] *liberty_box has quit (Ping timeout: 2m30s) ● Jan 09 [19:20] *liberty_box (~liberty@suig26pxj59pi.irc) has joined #boycottnovell [19:21] *DaemonFC has quit (Ping timeout: 2m30s) [19:26] *DaemonFC (~daemonfc@c6f2nzqiwd2bc.irc) has joined #boycottnovell ● Jan 09 [21:53] *tech_exorcist has quit (Quit: Disconnecting) [21:59] Techrights-sec back [21:59] Techrights-sec afk ● Jan 09 [22:53] *u-amarsh04 (~amarsh04@joseon-rmogvn.g0d7.dtdf.mc4289.IP) has joined #boycottnovell [22:53] *u-amarsh04 (~amarsh04@6pxtd49npuduw.irc) has joined #boycottnovell ● Jan 09 [23:09] *wallacer has quit (Ping timeout: 2m30s) [23:16] *wallacer (~quassel@6bsu33ajs4zs4.irc) has joined #boycottnovell [23:32] *psydroid4 has quit (Ping timeout: 2m30s)