Extract/Create Mini DV Metadata and etc...?

[Undead Sega]

As the title explains, i was wondering if it is possible to first extract the metadata from a Mini DV tape, which would contain all the information like dates and the timecode, as well as relating to timecode, i believe NLE's uses this information to help scene detect thus be able to capture a whole DV tape but having many small avi files for each shot or scene.

is it possible then to extract this information?

in addition, can one create this sort of data to help enable a new DV tape with scene detect when capturing? like place points where the scene starts and ends?

im pretty sure there were people who manage to do this, but i couldnt delve in any further, hopefully you lot can shine some light on me for this?

[Safesurfer]

The following apps can all get DV metadata to varying degrees.

Enosoft DV Processor

DVDate

DV_Datecode

[Undead Sega]

ahhh Excellent, however will i be able to extract it as a file or like a text/log file?

also can i create a metadata for scene dection when i am applying new video onto a fresh tape that of course doesnt contain this information?

[Safesurfer]

ahhh Excellent, however will i be able to extract it as a file or like a text/log file?

also can i create a metadata for scene dection when i am applying new video onto a fresh tape that of course doesnt contain this information?

DV_Datecode and DVDate (CaptureFlux is another app by the same author that lets you see the metadata on capture) both have .txt file output for the Datecode info. What do you want to do with this information?

I'm somewhat confused by your "Create metadata" question, your camcorder automatically creates this info when recording. Most capture apps use the datecode info for detecting scene changes by the changes made when you start/stop camcorder recording.

[Undead Sega]

okay then, what i meant by create metadata or whatever, is that, say u use ScenalyzerLive to do your scene detection and that itself creates a file with all the start and ends of each scene in the footage, and to take that information into whatever program that will implement itself when recording back to a DV tape.

[Safesurfer]

So you want to preserve the original DV metadata (i.e. datecode etc) after editing and outputting back to tape on your camcorder?

I know my video editor (Editstudio 6) can do this by "smart rendering." Only areas where effects/transitions have been applied are rendered and the data at those points is lost, otherwise the data is preserved. What editor do you use?

[Undead Sega]

i use Adobe Premiere Pro 1.5 and CS3, which im pretty sure does not preserve the metadata.

what i intend to do is both, yes, to preserve the original DV metadata, thus like taking the whole DV tape onto the computer and back onto tape, with its original metadata.

and the other was to write a metadata for newly existing footage that either lost or didnt have such metadata in the first place.

[Safesurfer]

OK.

I don't have Premiere Pro or CS3, so don't know if they are capable of smart rendering DV, maybe someone else can chime in on that.

DVDate does have the ability of writing a new datecode to a file :-

Overwrite a datecode: This command displays a panel where you may choose a date and time of start of the selected video (presuming it has format DV).

When you click on OK, the command creates a new file (in the same folder unless you have set otherwise in the preferences ) called like xxx_datecode.avi if the original file was called xxx.avi. This new file is very similar to the original one, except for the datecode written in each frame. The quality of video and audio is strictly unchanged.

The Enosoft DV Processor also allows you to change the Recording Date, Recording Time, and Timecode in it's Embedded Data Processing option. Have a look at the manual for full details.

[Undead Sega]

oh thats great stuff to hear!

well, with DVDate, when wanting to write a Datecode, do u have t manually input shot for shot? or can you take a .txt file and import it as the new datecode?

eitherway, can any of the programs do that on what i just mentioned?

[Safesurfer]

oh thats great stuff to hear!

well, with DVDate, when wanting to write a Datecode, do u have t manually input shot for shot? or can you take a .txt file and import it as the new datecode?

eitherway, can any of the programs do that on what i just mentioned?

No, I don't think either let you use a .txt file, from what I've seen you can only set a start date/time and it writes the data from that point. I'm not aware of any software that will let you do that.

[Undead Sega]

i wasnt just referring to .txt file only, i mean any other texts files made, would that now change ur response to that?

okay, if i have a set of clips that is all recorded back onto tape, i want each of those clips to be scene detected, will i be able to make a metadata, timecode whatever out of that?

[Safesurfer]

Not that I know of, I'm not an expert. Maybe Johnny Malaria at Enosoft would be able to advise more on what is possible.

You should be able to modify the datecode metadata of edited files with either of these apps. A quick search also turned up this app which looks interesting, but again I don't believe you can use an external file to modify the data.

http://www.dvmp.co.uk/

[Undead Sega]

i see, i look forward to him coming here

just to try and simplify what i am trying to do, say for example i have 3 serpeate clips that i will put together onto a DV tape, however, when i recapture them back onto the computer, i want to have it captured as 3 seperate clips as i had them originally.

beforehand of all of this, isnt there a way from the 3 clips to create a metadata or datacode so that it will know at the end to detect 3 clips, or scenes from the information made?

[JohnnyMalaria]

i see, i look forward to him coming here

H-e-r-e-'-s J-o-h-n-n-y

Is this what you are trying to do:

You create 3 clips with your editing software but this destroys the existing metadata (except for those frames that don't need rendering)

You send each of these to a DV tape (or join them before sending them as one clip)

At later date, you recapture the tape and what to recreate the original three clips but because the metadata were destroyed, you can't tell where the original clips start and stop.

If so, our Enosoft DV Processor can help you. It can write new metadata to the file which will persist when you write to tape. You can use the current time (from your PC's clock) to create new date and time. There's no need to change the timecode since your camcorder will always change it. The software can do it on the fly while writing a file to tape but because the camcorder is a mechanical device you may end up missing/duplicating frames between scenes. So, you should create a new, single file made up from individual clips for which you have already changed the metadata. Once that is done, you write this new file to tape. Because the metadata in each new clip will be discontinuous (if you use the computer's clock for the new metadata), scene detection software will successfully identify the clips when you recapture.

Here's the process:

For each clip, open it in our software, configure it to write new date/time based on the clock (see manual) and create a new file. There's no recompression involved - it just changes the metadata. It is important to run this in real-time (the default) so that the date/time changes at the right speed.

You can create the new clips as AVI files or raw DV files. If you choose to use AVI files, you need to find some other program to join them together to make the single file. Windows Movie Maker will do this (it will just copy the frames as-is and preserve the new metadata). Just drop all the clips onto the timeline and render as a new file. If you choose to use raw DV files, you will be able to join them together very easily from the command line using copy /b clip1.dv+clip2.dv+clip3.dv clipjoined.dv

Either way, once the single file has been created, use our software to send it to tape.

[Undead Sega]

hello Jonny, that is great to hear! and this will seem to help me greatly as well!

however, when i mentioned 3 clips, that was merely an example, because i have created footage that lasts up to an hour or more, and of course this has more than a hundred scenes and that will be sent to a DV tape. This footage is newly created and of course will have no metadata whatsoever.

therefore, i will using an Optical Scene Detection software to help me split the scenes and somewhat export a file or list of all the scenes that starts and stops (thats if i can find an app that can do this).

i guess u can figure it from there, but may i ask does Enosoft DV Processor do CSV file? export and import?

[JohnnyMalaria]

Our software can be controlled by many other programs. Technically, it has an extensive COM interface that, in practical terms, means software such as Excel, anything written in VB .NET and even VBScript can control it. The latter is, in effect, a glorified batch file that you can create in nothing more complicated than Notepad.

As long as you can provide the clips as separate files, the whole thing (writing new metadata, joining the files together and sending it to tape) can be done from a single script file probably just a few dozen lines long.

If, on the other hand, you have one large file and a list of edit points, that will require something a bit more capable. It could be done as an Excel macro (i.e., a VBA program). Our software comes with an example of just what can be achieved with Excel + our software, including creation of a scene list from a tape while being captured. For you need, you would have to provide a file and specify the offsets from the start of the file for each clip (e.g., in frames or seconds). The software would add new metadata to each clip and join them together before sending to tape.

I'm happy to put something barebones together (at no cost assuming this is for personal use )

Do you have Excel (or any other Office application) or a compiler for VB .NET (or VC++)?

[Safesurfer]

I did a quick search on the forum and found this image of the export settings for Premiere. I would be willing to bet that unchecking the Recompress check box in the Video settings would "Smart render" the DV and preserve the metadata except at transition points or where you've applied effects that require reencoding the video. It would be worth testing and could then remove the need for recreating metadata on exported clips.

[Undead Sega]

this is really great! but may i ask, can your software be able to do Optical Scene Detection? if not that fine, as i will use either Scenalyzer or MSU Scene Change Detector which that itself outputs a .CSV file which opens in Excel (which i do have infact )

so techncically, if i have that file, with all the information where a scene starts and stops, could i take that into your software with the one large avi file, and thereofre it will know afterwards when recapturing where to split the files?

as for Premiere Pro, i always uncheck the Recompress checkbox, because even though i didnt know it will preserve the metadata, i feared it will recompress the quality itself.

[JohnnyMalaria]

Our software won't do optical scene detection (we had something years ago but the NLE it supported ceased to be).

Won't Scenalyzer split the file for you? It might be easier that way.

An Excel macro could be written to use your CSV info and control our software to change the metadata. Can you send me (or post here) an example CSV file so that I can test it with one of my DV files?

Re PPro - do you do any processing on the timeline or do you just do simple cuts? If the latter, the rendered file will do the job. As soon as you process just one frame then you have to go through the whole process we have described. Also, doesn't PPro export edit decision lists? i.e., a simple text file with a list of in/out points for each clip on the timeline. This will remove the need to do the optical scene detection.

[Undead Sega]

you are correct, Premiere Pro does export an EDL, which i have myself a couple of times. however my 1-2 hour footage would have to be split before exporting/

my CSV file would be made through MSU Scene Change Detector, im not sure it will be in the same format.

[JohnnyMalaria]

OK.

The format wouldn't matter as long as it is simple to retrieve the information.

[Undead Sega]

i see then, well the MSU Scene Change Detector is doing me quite well as well as producing the CSV file.

probably last question about your software, when taking footage back onto DV tape, does the file have to be a DV-AVI file already? or does it encode it to DV during the process?

also, the footage is both mixed 16:9 and 4:3, so when taking to DV tape with Premiere Pro, u can interpret the footage to the correct aspect ratio, so when capture back to a computer, the file itself will play correctly, same goes with the 4:3 settings. Can yours do this if u know what i am talking about?

[JohnnyMalaria]

probably last question about your software, when taking footage back onto DV tape, does the file have to be a DV-AVI file already? or does it encode it to DV during the process?

It accepts DV format only (AVI or raw). Won't you be rendering to DV from the PPro timeline anyway?

also, the footage is both mixed 16:9 and 4:3, so when taking to DV tape with Premiere Pro, u can interpret the footage to the correct aspect ratio, so when capture back to a computer, the file itself will play correctly, same goes with the 4:3 settings. Can yours do this if u know what i am talking about?

Let me make sure I understand - you will create a single file from PPro that is, in effect, lots of clips joined together and you will create a CSV file to indicate where the cuts are. But some of the clips are 4:3 and some 16:9. How can you render a single file from PPro with mixed aspect ratios? It is certainly possible for our software to change the metadata such that it also sets the aspect ratio as needed but there's no way for it to tell what it should be just from the video. You need to add that to the CSV file. You may want to consider separating your workflow into just 4:3 and just 16:9 portions.

[Undead Sega]

You may want to consider separating your workflow into just 4:3 and just 16:9 portions.

yes, i thought of that as well and did that before, therefore i was thinking for this project, split the video into different AVIs according to their aspect ratio and do scene detection on each of them.

so i would have one video that is 16:9, would the software correct/flag the video to its aspect ratio? and do i have do it one by one, as in take each video clip along with the CSV file?

[JohnnyMalaria]

Yes, it would. PPro *should* set the aspect ratio flag in the DV file according to your project settings. If not, it would be easy to add it while changing the data/time etc. You could simply name your single files in such a way to differentiate them - e.g., MyFirstFile169.avi vs. MyFirstFile43.avi. The Excel macro (if that's what we use) would use the names to set the right aspect ratio. Either way, you'll need to process each file separately. Once complete, all the files could be joined into one before sending to tape.

[Undead Sega]

okay, so if i were to take my 16:9 DV file back to tape with your software, would i use the CSV file at the same time or afterwards?

[JohnnyMalaria]

You'd create the CSV file with your optical scene detection software and use it to specify where to change the metadata within the file. Assuming PPro creates the original file with the correct aspect ratio, it will be recorded to tape with the correct aspect ratio flag. Your tape would end up with a string of, say, 16:9 clips followed by 4:3 clips. When you recapture the tape, the aspect ratio flag will be correct.

[Undead Sega]

i think i wasnt clear what i was trying to say, how does your software work with the CSV file when backing ur footage to a DV tape?

[JohnnyMalaria]

Any way we choose, pretty much. Because an Excel VBA program will be needed to control the processor (in order to change the metadata data etc), it will have to parse the CSV file. Without seeing an example of the CSV file, it's hard to say exactly. It might be as simple as inserting a new line at the very top to specify the aspect ratio for that particular file. Can you post an example?

[Undead Sega]

here is an example of what MSU Scene Change Detector, it should be an attachment here.

also, i would like to ask, when taking the CSV file which is the modify the metadata, am i doing it on the tape? or on the DV file itself to which i will be taking it to the tape? do i modify the DV-AVI file or on the tape? or am i getting the picture wrong?



scd_yyuv_plan%20a%20no%20limits%20footage%201%2016-9.csv