Go Back    Forum > Digital Video > Video Project Help > Capture, Record, Transfer

Reply
 
LinkBack Thread Tools
  #1  
03-31-2019, 10:08 PM
jwillis84 jwillis84 is offline
Free Member
 
Join Date: Aug 2017
Location: College Station, TX
Posts: 800
Thanked 217 Times in 174 Posts
A small update.

Progress on the TFS2 file system which exists on the Polaroid 2001g, Toshiba RD-XS64, and many JVC MX, MH or DVM recorders has slowed to a stop.

We just couldn't make much progress towards understanding it, and its not of much use without mating the recorder title and stitching all its pieces back up.. and in order.

If anyone knows anything about this format please contact me, it would be very helpful.

It appears LSI Logic created it as part of its CoreWare suite in 2001 after acquiring C-Cube to produce the DominoFX chipset, which was the consumer side of the Ziva chipset for broadcasters.

After the LSI Logic chipset patents and licenses were sold off or spun off into Magnum, the trail grows very cold. Magnum didn't last long and appears to have simply folded. They only marketed one successor to the DominoFX and I don't know if it was ever used in any product before they disappeared.

The DVR layered a special file and directory system on top of the TFS2 partition layer, and any information regarding that would also be helpful.

Only one person looking at the Salora in Belgium or Finland ever made any head way.. and he ceased looking at it about a decade ago.

Instead we're looking at the Pioneers and Panasonics, which are very different indeed. Still making progress.. but nothing is for sure. The PNR format is new as well.. different from anything else seen.. but a little less stressful than looking at the TFS2 and LSI_DVR_HDB0 formats.

On the bright side made a lot of progress on the SDHC and Compact flash replacement of hard drives, across many brands and models.. too much to post about here.. and probably not of general interest.

The idea is basically to provide a way of supporting all of each recorders features, even if the burner dies, and providing a way to continue using those recordings, on the current DVR, a different DVR or on a PC.
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
04-16-2019, 02:16 PM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,501
Thanked 2,447 Times in 2,079 Posts
If anybody can ever help on this, please post.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
  #3  
01-05-2020, 12:32 AM
vprevelakis vprevelakis is offline
Free Member
 
Join Date: Jan 2020
Posts: 3
Thanked 1 Time in 1 Post
Quote:
Originally Posted by jwillis84 View Post
A small update.

Progress on the TFS2 file system which exists on the Polaroid 2001g, Toshiba RD-XS64, and many JVC MX, MH or DVM recorders has slowed to a stop.

We just couldn't make much progress towards understanding it, and its not of much use without mating the recorder title and stitching all its pieces back up.. and in order.

If anyone knows anything about this format please contact me, it would be very helpful.
Hi,

There was some work in reverse engineering the format by jeroent back in 2008. The site is now dead but you can get a copy from the wayback machine (https://web.archive.org/web/20110913041107/jeroent.com/salora/)

**vp

www.series80.org
Reply With Quote
The following users thank vprevelakis for this useful post: jwillis84 (01-05-2020)
  #4  
01-05-2020, 01:32 AM
jwillis84 jwillis84 is offline
Free Member
 
Join Date: Aug 2017
Location: College Station, TX
Posts: 800
Thanked 217 Times in 174 Posts
Thanks for trying.

I believe we did try to contact him as he is from the same country as Peter and he speaks his native language. But attempts to contact went unanswered.

Not to get all cyber stalker, but I also think we did find his other interests online and I think he has moved on as an educator and probably no longer recalls this work.

The programming manuals for LSI chipset apparently leaked online and are out there today. They have a standardized API for using the chipset with any C programming language.

It appears the DominoFX suite was designed around a core of two SPARC processors, one used for real-time video manipulation and the other for multi-tasked user interface manipulation.. sharing access to the same resources in a single chip. As a low level implementation of sophisticated programming tasks in on-chip silicon, the instruction words for the chip had instructions for "streaming" the video and audio straight to the ATA/IDE hard drive.

This sounds like stories about the TiVo approach to storing video on their hard drives.. which placed a premium on simplifying the task of data storage for the silicon's benefit. That is the microprocessor doing the video capture determined the on disk storage format.. not the firmware or the software on the recorder.

These did not store the video in the "published" standard DVD-VR format, or any variant of that.. even though the TFS2 does have some similarities to DVD-VR in its Title list meta data. That is usable, but it doesn't relate to where and how the video is stored on disk.. except perhaps in a vague simple integer position way.

Rather it appears TFS2 stored recordings in a streaming "buffer" format which incremented backwards, counting down from the end of the hard drive towards the beginning. This might have been so one SPARC processor focused on the "end" of the drive and incremented backwards to maximize usage of any size hard disk. Larger drives would have simply had further to go before running out of space. While the other SPARC processor focused on the "opposite end" of the hard drive.. say.. for user interface scheduling.. once set.. that end of the hard drive would not be used very much.. and so they pretty much stayed out of each others way and did not end up fragmenting each others hard disk "task space".

This is unpleasant.

It slows the progress on determining precisely "how" the silicon behaves under different circumstances and what it does to the hard disk.

Many, many other brands and models simply adopted DVD-VR or a kind of straight forward version of it, making search algorithms that "discover" the brand and model based on their variant of the standard pseudo-easy to support quickly. Once a "fingerprint" is found.. solid support is mostly a matter of testing.

Polaroid TFS2 was derived from a "kit" provided by C-Cube and LSI, as were many based on the DominioFX chipset.. but JVC seemed to take it in a different direction.

Last edited by jwillis84; 01-05-2020 at 02:15 AM.
Reply With Quote
  #5  
01-05-2020, 11:46 AM
vprevelakis vprevelakis is offline
Free Member
 
Join Date: Jan 2020
Posts: 3
Thanked 1 Time in 1 Post
Hi,

Thanks for the tip regarding the manuals, I downloaded the DMN-8600PG_vol1.pdf and DMN-8600PG_vol2.pdf files and looked at them and as you point out, there is very little info on how the bits are actually arranged on the disk.

However in the "Hacking the Salora HDD2510" document by jeroent there are hints as to the layout. Unfortunately my DVR (a Yamakawa DVR-645) is no longer working so I can't experiment with its drive.

Initially my intention was to follow in jeroent's steps using a blank 2Gb disk, and take snapshots of its layout as videos were added to it. However, I did not get very far before being sidetracked and then the DVR stoped recognizing the disk and my efforts ground to a halt.

At the time I had modified the DVR so that it had an external bay for the hard disk, so that I could swap disk drives. So I have a pile of hard disks with video material that I cannot recover.

Now it appears the electrolytic capacitors used by Yamakawa were not defective, so there is the possibility of fixing the DVR and trying once again to experiment with it.

In the mean time, perhaps if one who already has a working DVR can send me some disk images: e.g. a newly formatted disk, then the same after adding one video, the same with two videos, and finally after deleting the first video) I'd be able to work in parallel with the Yamakawa repair.

Best Regards

Vassilis

www.series80.org
Reply With Quote
  #6  
01-05-2020, 11:56 AM
vprevelakis vprevelakis is offline
Free Member
 
Join Date: Jan 2020
Posts: 3
Thanked 1 Time in 1 Post
I forgot to add that the best video would be something like the recording of a running stopwatch, to provide a time reference for the frames.

Thanks

Vassilis

www.series80.org
Reply With Quote
  #7  
01-05-2020, 04:22 PM
jwillis84 jwillis84 is offline
Free Member
 
Join Date: Aug 2017
Location: College Station, TX
Posts: 800
Thanked 217 Times in 174 Posts
As requested, a recording of a signal source that has a Time and date clock that increments in real time.

attached is a hard drive image from a Polaroid 2001G/[B model]:

4GB SINTECHI HighSpeed SD.zip

I found that this recorder could use SD cards rather than hard drives with an SD to IDE adapter. So I could use much smaller size SD cards to produce hard drive images that were much easier to format, backup and share over the internet in their Raw byte format than actual hard drives. The images are identical to how it formats hard drives except for the fact the recorder detects there is much less total storage space for recordings.

Isobuster can backup the Raw byte format of the SD card to a file, or the tool "HDD Guru - HDD Raw Copy Tool" can do the same, they can also restore the Raw byte format image to SD cards or hard drives as needed.

In the Netherlands there is a tool that searches specifically for MPEG2 frame characteristics called "Defraser" it has no problems finding the LBA of the MPEG2 recordings. However it can be a challenge to learn how to use. Its free for personal use but requires a license for commercial use.

This is an example of what it finds in the SD image linked above:

4GB SD card one recording in Defraser.jpg

The Raw byte image is small enough that when Defraser is run against it, you can see it processing in the lower right corner of the application window with a progress bar.. from beginning to end. Normally disk images are so large this bar is glacially slow and you never notice it. It corresponds to position within the image. It does not begin detecting frames until almost the end of the image.. confirming the position of the recording as being at the far end of the drive image.

This is not a real "solution" for extracting "anything" since most MPEG2 DVR recorders are fragmented and jumbled up and do not associate "Titles" or any other meta data with the frames that are found. Its just a neat GUI tool for locating frames within the image and confirming they are not encrypted.

In fact between jeroent and Peters work all of the meta data (except) for position of the frames on the Polaroid 2001g/[B model] have been figured out. The position (might) be as simple as the implicit integer number associated with the meta data.. although how the recorder deals with fragmentation could be an issue for later.

This is an example of grabbing all of the frames in Defraser discovered and "tossing them" into a file:

4GB Frames Thrown in a File.m2v

Really I would caution anyone trying to use Defraser that there is no "Users Manual or Users Guide" (exactly). I think it was intended for use by Law Enforcement for the Netherlands and targeted HDD camcorders or Phones.. using it in this way is bending its original usage model into twists like a Pretzel.. so explaining how to use it in this way might be a good topic of conversation some time.. but simply these files demonstrate.. this is not an impossible task.. just difficult. And I'm not the best at solving really complex binary logic puzzles.

For reference:

I use Windows 7 x64 to run Isobuster, Defraser and playback the tossed frames in Windows Media Player and VLC.

Isobuster and Defraser work fine on anything from Windows XP 32 bit up through Windows 10 and on Mac OS X - Intel with vm, they are fairly isolated from the underlying OS. But Isobuster happily uses the USB pass-thru subsystem to directly connect to IDE drives from any OS. That's another topic of conversation some day.. but this reply is far too long as it is.

addendum:

And here is an example with the audio bits tossed into the salad. Again, this is using Defraser but selecting the frames in a (different) way.

4GB Frames (and audio) Thrown in a File.mpg


It really frustrating to be so close and not get back to this for almost a year.

TFS2 is a pain.


Attached Images
File Type: jpg 4GB SD card one recording in Defraser.jpg (78.3 KB, 11 downloads)
File Type: jpg 4 GB vlc media information during playback.jpg (32.4 KB, 1 downloads)
File Type: jpg 4 GB video redo during playback.jpg (86.3 KB, 3 downloads)
File Type: jpg 4 GB scopes during playback.jpg (65.3 KB, 4 downloads)
Attached Files
File Type: m2v 4GB Frames Thrown in a File.m2v (11.75 MB, 4 downloads)
File Type: zip 4GB SINTECHI HighSpeed SD.zip (16.44 MB, 4 downloads)
File Type: mpg 4GB Frames (and audio) Thrown in a File.mpg (12.78 MB, 9 downloads)

Last edited by jwillis84; 01-05-2020 at 05:22 PM.
Reply With Quote
  #8  
11-16-2020, 02:24 PM
dpeddi dpeddi is offline
Free Member
 
Join Date: Nov 2020
Posts: 4
Thanked 0 Times in 0 Posts
Hi guys,

I'm having quite success with recovery video from my Packard Bell EHR2080 using following software with some patches and a good setup.

https://github.com/haliner/dvr-recover

Probably my configuration an be improved... but parsing 80gb of disk with this tool seems quite a success

- The setup:

Code:
E:\>c:\Python27\python.exe dvr-recover.py setup
input_file: ehr2080.dd
export_dir: e:\dvd-recover-export
blocksize: 2048
min_chunk_size: 6400
max_create_gap: 90000
max_sort_gap: 90000
- The patch:

Code:
if self.current_block is None:
            if self.db_manager.chunk_count() != 0:
                raise CreateError('No state information, but chunk '
                                  'count is not 0. Probably the scan '
                                  'finished already. Abort process to '
                                  'avoid loss of data. Use parameter '
                                  'clear to clear database (you will '
                                  'lose all chunk information).')
-            self.current_block = 0
+            self.current_block = 0x9857F

-        self.reader.seek(self.current_block * self.blocksize)
+        self.reader.seek(self.current_block * self.blocksize + 0x600)
For the rest you can read the original documentation...

Probably with the command is quite easy to recognize all the slices with the address, and probably by looking for these address in the TFS2 TOC it would be possible to recognize all the format.

Code:
E:\>c:\Python27\python.exe dvr-recover.py show|more
-----+--------------+--------------+--------------+--------------+--------------
     |  Block Start |   Block Size |  Clock Start |    Clock End |  Concatenate
-----+--------------+--------------+--------------+--------------+--------------
Regards
Reply With Quote
  #9  
11-17-2020, 06:17 AM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,501
Thanked 2,447 Times in 2,079 Posts
Quote:
Originally Posted by dpeddi View Post
I'm having quite success with recovery video from my Packard Bell EHR2080
But is it a TFS2 file system?

The page you linked to was for the Panasonic file systems, which are simple by comparison. So should I assume that the Packard Bell is a Panasonic clone?

I see a few images online. It looks somewhat like a JVC, but also not at all.


Attached Images
File Type: jpg Packard Bell EHR2080.jpg (6.2 KB, 4 downloads)
File Type: jpg Packard Bell EHR2080_2.jpg (12.0 KB, 4 downloads)

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
  #10  
11-17-2020, 08:02 AM
dpeddi dpeddi is offline
Free Member
 
Join Date: Nov 2020
Posts: 4
Thanked 0 Times in 0 Posts
Packard Bell ehr2080 is based on LSI.

I posted here the modification to make it work with my packard bell.

Note that the approch is brute force, it doesn't really analize the fs but just the mpeg timestamp
Reply With Quote
  #11  
11-17-2020, 08:05 AM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,501
Thanked 2,447 Times in 2,079 Posts
Quote:
Originally Posted by dpeddi View Post
Packard Bell ehr2080 is based on LSI.
I posted here the modification to make it work with my packard bell.
Note that the approch is brute force, it doesn't really analize the fs but just the mpeg timestamp
Please PM jwillis, get his attention, and you can both discuss more in this thread.

This bodes well for my Polaroid LSI and JVC LSI.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
  #12  
11-19-2020, 04:09 PM
dpeddi dpeddi is offline
Free Member
 
Join Date: Nov 2020
Posts: 4
Thanked 0 Times in 0 Posts
more analysis:

I've noticed by comparing 4gb hdd and 80gb hdd that some the TFS2 blocks starts at different addresses.
The blocks that contains str listing have a dynamic size. It seems that this block size can be calculated (@2060653E-F)

0x10004000 '\0'
0x200 CRedundantAbs
0x800 CLsiPMABS
0x200 CRedundantAbs
0x800 + 0x10600400 CLsiPMABS (vtm/ttm/titles with TFS2 header)
0x200 CRedundantAbs
0x800 CLsiPMABS (with TFS2 header)
0x200 CRedundantAbs
0x800 CLsiPMABS + 0x200 +800 * 200 + [value @0x13e<<8 + @0x13f] * 0x400 (str with TFS2 header) {on 4g image is 0xcc}
0x200 CRedundantAbs
0x800 CLsiPMABS (with TFS2 header)
nn * 0x100000 blocks [...]

Does it exists some public source code other then the one from jeroent?

E.

Last edited by dpeddi; 11-19-2020 at 04:53 PM.
Reply With Quote
  #13  
11-19-2020, 05:49 PM
dpeddi dpeddi is offline
Free Member
 
Join Date: Nov 2020
Posts: 4
Thanked 0 Times in 0 Posts
attached the modified dvr-recover with computation of the start of mpeg data


Attached Files
File Type: zip dvr-recover.zip (9.2 KB, 11 downloads)
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Understanding PAL aspect ratio? spanak Encode, Convert for discs 4 04-17-2018 02:16 AM
Understanding and setting brightness for capture jimmyz80 Capture, Record, Transfer 10 08-20-2015 01:49 PM
Understanding HDR and Exposure Stops JasonCA Photo Processing, Scanning & Printing 5 03-04-2013 11:04 AM
Improving my understanding of video and its formats Sossity Videography: Cameras, TVs and Players 1 09-28-2012 08:57 AM
Understanding hardware strangepork Computers 1 10-19-2004 05:42 AM

Thread Tools



 
All times are GMT -5. The time now is 04:27 AM