It’s early to tell if all the goals of Treasured have been met, but for sure the landscape has changed for me:
Ten to twenty times more people contact me for a movie repair now. Treasured is definitively the hook that my Movie Repair Service needed.
But two thirds of them are not actual customers, they are just playing with the app and uploading a trashed DivX or WMV file.

The remaining third is a mix of my traditional customer base of video professionals, and a new category of customers. By lowering the barrier to request a repair, I’m attracting new types of repairs, new types of files, for which Treasured was not initially designed or optimized.

Bottom line: My claim of 90% accurate diagnostic, 80% with preview, that I religiously checked before Treasured was released, is wrong due to the diversity of movies I’m seeing now.

I can imagine some Apple execs running frantically in the hallways and meeting in an underground war room in Cupertino headquarters. In the dim light, they start analyzing Google Chrome features (of lack thereof) and suddenly Steve slams the table with his fist and yells: “Safari is dead if we don’t change the direction now.”

Google Chrome must have been a shock for Apple’s Safari team. Apple developers were probably quietly completing alpha stage for Safari 4.0, and suddenly, bam!

Now Safari must redefine itself, and go in a different direction, and do it fast. SquirrelFish Extreme, announced today, only a few weeks after the original SquirrelFish Javascript engine, is maybe a sign of things starting to move. If SquirrelFish was a leap forward, why announce another leap forward before even releasing it? Because of Chrome, imho.

Every day, I spend a few hours on a Windows machine, and this week I’ve noticed that I’ve stopped using Safari completely. Chrome is my browser of choice in Windows now. A threat for Safari on the Mac, for sure.

Apple will probably not conform with playing catch up, and I’m looking forward to see what exciting stuff there are preparing for January. Or maybe Safari 4.0 has to be delayed, but for good.

Where I had twenty programs capable of repairing one type of damaged movies each, now I have one Frankenstein creature capable of previewing them all.

Note that I’m not saying repairing, just previewing. Because what I’ve grown in scope, has come at the cost of feature depth.

I’m happy with that, because it’s exactly what I wanted to do:
Being able to view what is inside a damaged movie file is indeed very important. It’s telling you two things:

• First, that your beloved footage is still here, latent, waiting for a repair. You can almost touch it.
• Second, that there’s someone that cares about video professionals. It’s not always a fairy tale, shit happens. And it’s good to know that someone is here to help.

The reason why Ventilo does not repair the movie is a technical one: If getting a computer to analyze the contents of a file and preview a cherry-picked frame is a hard task, it’s an easy one compared to extract all the frames, video and audio, and present them perfectly synchronized, without artefact, in a broadcast quality movie.

Preview is a feature that luckily falls at the tiny intersection of what people wish could be done, and what technically I’m able to do. And it’s also the missing piece of the Movie Repair puzzle because it’s what creates the emotion, the engagement.

Ventilo was born from an intuition:

Every day, a few hundred (unlucky) people discover that an important movie file, because it’s urgent or because it’s a piece of their memories, has been damaged and cannot be open. They seek help in friends, colleagues, Internet forums, and end up searching for a “tool to fix damaged movies” or a “software to repair corrupt film”.

What they find is a dozen of crappy products that will lead them nowhere. The problem is that quality movie repair is delivered as a service, not as a software. But people doesn’t look for a service, it looks like a strange idea.

So how can I let them know, and help them recover their movie?

Enter Ventilo. A free tool that they will easily find and use, and that will tell them the truth about their file:

“No, it can’t be repaired because the file contains only alien data”

“Yes, it can be fixed, look at the preview… but the repair is a complex task, it can only be done by a video hacker and this service can cost you between x and y.”

Ventilo also facilitates some hardeous tasks, like data extraction and transfer, and saves time both to the customer and to the repair technician.

In summary, Ventilo lowers several barriers:
Barrier to discovery (Google will find it), to use (it’s software), to gratification (free preview!), and to engagement (big hope, small time footprint).
Thus enabling many more people to get their file repaired.

As I’m working alone, instead of a specification, I prefer a small set of guidelines that let me decide quickly whether something must, can, or cannot be added to Ventilo.

My guidelines:

• Tell something interesting about all the files in my “collection” of corrupt movies, and preview 90% of them.
• Release 1.0 within one month, stable enough.
• Provide a simple user interface that makes Ventilo feel like a Mac application.
• Give an emotional touch to the Preview.

Ventilo is far from finished, the user interface work has just started. I’ll share with you in the next days the first screenshots and discuss a couple of interaction design items.

click to enlarge

Contact me if you would like to beta-test Ventilo. bjoossen at aeroquartet.com

When you repair movies, you never know what you can find.
Yesterday I had an old shoe moment while looking for some mountain footage in a corrupt file.
Some obscure AVI Windows file containing this infamous “Download” animation surfaced when I least expected it:

Today I’ll teach you to fish, in other words I’ll explain how to rescue movies from inside larger files. It will only work if certain conditions are met, but it’s an exciting experience… that can end with an old shoe. You’ve been warned.

Container Structure Correction is a repair technique that acts upon the container structure data, leaving the media data and the index and tables untouched.
Unlike reindexing or other techniques that need to act on hundreds of audio or video frames, a structure correction is usually a question of few bytes to correct or a misplaced block of data. For this reason, it can be done manually.

If the same correction has to be applied on a collection of files, then maybe it’s worth spending time automating the structure correction.

Most common structure damages and how to fix them:

Full movie data, including container, is embedded in a larger file, for example after a data recovery.
In this case, the data can be extracted and saved with the container suffix and it usually works.

In this example, we have examined the file with an Hex editor and have found an intact RIFF structure in the middle of a file. Note that the length of the structure 0×49e4 is encoded just after RIFF in 32 bits, big-endian.
A quick check shows that after 0×49e4+8 bytes, there is suddenly no data, thus confirming that we are into something.

By copying this data to a new file and saving with an AVI suffix, we now have a valid movie.
Make sure you copy exactly the required bytes, starting from the R in RIFF, ending after idx structure, otherwise it won’t work.

When you edit a movie in-place (ie without writing again all the file), some edition softwares just add a new moov container at the end of the file, without bothering to remove older ones. If the file has become corrupt, those old containers can still work.
The trick consists in finding the moov structures and redirect towards an older one.
You’ll need an Hex editor and good knowledge of QuickTime file format to do that.

Files from a data recovery can present defects: contents not matching the file name, truncated files, mashed contents coming from several files, moved or missing blocks.
As data recovery acts at the lower filesystem level, the data is always presented in blocks, for example exactly 2048 bytes.
A movie can fail to open because some blocks have been moved, duplicated or missing. The fact have blocks always have a length of 2048 bytes helps to detect and correct those problems. It’s like solving a puzzle and is usually complex.

I feel identified with this description of independent Mac developers. Hopefully I can join this ‘creative class’ soon.

Indie Mac developers appear to be prototypes of what Richard Florida (2002) calls the ‘creative class’. On average they do not make a sharp distinction between their personal and professional lives (Florida, 2002, p.14), they appreciate systems of meritocracy and subscribe less to the status of wealth, they seem more motivated by the respect of their peers than by money (idem, p.78) and rally around their professional identity as a software entrepreneur (idem p.80). Indies fall into the category of ‘free agents’ (idem, p.107): people whom willingly sacrifice the securities of a job to be able to pursue their own creative interests.

But you can only pursue creative interests full-time once you’ve solved the money problem. Thus making remarkable products that sell is the most difficult part of the equation.

The “Chest”, my preferred one, was featured in the previous post. But a second idea also looked interesting and I decided to develop it as well:

A common problem I find when I write or design stuff, either an user interface, an illustration or an icon, is how difficult it is to judge your own work.

Usually, I let pass a few days before I look at it again. While you’re designing, you’re trying hard to make a certain message emerge out of the drawing, so your perception is biased until you’ve evacuated it completely. You can discover that the design doesn’t work or conveys a different message, but only after a while.

That’s why I’m leaving those two icons in this “almost-done” stage, until my mind cools down and I decide that I like them enough to finish the job.

As it’s just for a secondary application, that plays a small role in a bigger thing, I haven’t engaged the services of a professional designer.

I’ve still some rough edges and details to polish, but I’m pretty happy with that. It took me about 10 hours of work:
Brainstorming, selecting ideas, developing a few ideas, handmade sketches, mock-ups to see what works and what doesn’t, pick an idea, and finally do detailled drawing.

A few lessons learnt:

• You will end with something different than what you plan.
• Design at once Application Name + Icon + “Catchphrase”. Otherwise it’s too heavy for only one element to convey all the force of your idea. Make each element reinforce and complete the other two.
• No more than 2 or 3 visual elements in your icon. Beyond that, it will not be understood and will not work at small sizes.
]]> http://www.aeroquartet.com/wordpress/2008/08/18/new-icon-work-in-progress/feed/ http://www.aeroquartet.com/wordpress/2008/08/16/zip-test/ http://www.aeroquartet.com/wordpress/2008/08/16/zip-test/#comments Sat, 16 Aug 2008 13:24:24 +0000 Benoît Joossen http://www.aeroquartet.com/wordpress/2008/08/16/zip-test/ (Second instalment of “Movie Repair Guide”)

When confronted with a corrupt movie file, there’s a quick way to check that it still contains the media data, which is a necessary condition to repair it: the “Zip test”

We create a compressed version of the file, which tells us the compression rate. For example, 50% means that the zip file is only half the size of the original one.

Audio and video data will give a low compression rate whereas other type of data will be around 50% or higher.



In Mac OS X, you can use the zip utility, either from the Finder, or from the Terminal:

zip --filename--

In Windows, you can use WinZip utility.

Here are aproximate compression rates you get:

• For data completly erased (file full of zeroes), 90 to 100%
• For alien data, 25 to 100% (Office documents: 50%, Applications: 40%)
• For audio integer 16, 21%
• For AIFF, 17%
• For DVCHD Pro, 11%
• For DV, 5 to 12%
• For other video codecs, DivX with mp3, Intermediate, MPEG2, H264, JPEG: 0 to 8%

There is a big gap between audio/video data and any other type of data. If you have a corrupt file, and the “Zip test” gives you a result over 25%, probably a complete repair is impossible because the data is partially or completely gone.

You can also refine the test by measuring on segments inside the file:

With a 500mb file, we split in 5 segments of 100mb and run the zip test on each segment with this two Mac OS X Terminal commands:

split -b 100m corrupt.mov
zip deleteme xaa xab xac xad xae

(The first one, split, cuts the file in chunks of 100mb each, called xaa, xab, xac, xad and so on.)
(The second creates a compressed archive called deleteme from xaa, xab, xac, xad and xae.)

This gives for example:

xaa (deflated 5%)
xab (deflated 5%)
xac (deflated 12%)
xad (deflated 100%)
xae (deflated 100%)

Which tells us that the media data in last two segments is probably lost. If a repair is possible, maybe 50 to 60% of the original file will be restored.

]]> http://www.aeroquartet.com/wordpress/2008/08/16/zip-test/feed/ http://www.aeroquartet.com/wordpress/2008/08/13/expectations/ http://www.aeroquartet.com/wordpress/2008/08/13/expectations/#comments Wed, 13 Aug 2008 15:01:59 +0000 Benoît Joossen http://www.aeroquartet.com/wordpress/2008/08/13/expectations/ (First instalment of my “Movie Repair Guide” serie)

Not all repairs can be equally successful at restoring original quality and integrity of footage.

It’s due to three factors:

• a. initial state of data
• b. redundancy of data
• c. ability of the repairman


a. initial state of data
The worse case is a file that even if it keeps the correct movie name and size, no longer contains data related to the movie.
It can either be “alien data”, random data, and it can affect a part or the whole file.
It’s clear that no repair is possible here. I don’t do miracles and I will just tell you that it cannot be repaired (free of charge, of course!)

The best case is a file that contains all the media data, cleanly arranged in frames, but that fails to open because some small detail is not correct. If you correct this detail, the file is repaired at original quality. This detail is usually the structure of the container and the index of the media inside the file.

In the middle, you find files that still contain 50% to 99% of the media data, but it’s completely mixed and disorganized. Furthermore, the file can contain some quantity of alien data. Those files usually come from a failure in the file/operating system or in the storage device (Hard disk, memory cards, …). Even with the best repair skills at work, one cannot expect in this case to recover the entire movie at original quality. The cost of such a repair will be much higher than a “best case” repair, because the techniques involved are more complex and need customization to each case, and even if the results is far from “broadcast quality”.

Finally, initial state of data can be misjudged when the repair is developed on a sample of the file. The sample can be clean, but on average the whole file can contain several “hot spots”, pattern breaks and segments full of alien data. This can lead to an increase between quotation given after diagnostic and final cost.

b. Redundancy of data
A rule of the thumb is that the more advanced codecs you use, the more efficient compression you get. But at the same time, the repair becomes more difficult.
The reason is that efficient compression leaves few hints to do a repair, as efficiency means no information “wasted” to make encoding redundant and movie structure easy to read. The extreme case would be a codec so efficient that the bitstream looks like random data, and that any random bitstream is decoded as valid.
AAC audio codec, for example, is quite close to this situation.

Identifying an audio or video frame in such cases is really challenging, and a certain failure rate is to be expected. If you have 5% failure rate repairing AAC, the result is barely audible. At 0.5% you have something decent, but not broadcast quality.
On the other hand, in codecs like DV, the failure rate is 0 because more than 1% of the data is predictable and redundant, so mistake is impossible.

c. Ability of the repairman
An experienced video hacker with the right set of tools: That’s what I’m trying to become. If you are able to detect the subtle data patterns inside the damaged file, you have the key for a quality repair.

Below, you see the same movie frame repaired by an experienced hacker (on the left) and by an inspired hacker (on the right, also me ;). This was a very tough repair, because we were on the bad case for initial state of data. This explains why a traditional repair leaves a lot of artifacts on the picture. But after more work, I discovered a more advanced technique, called de-spotting, that eliminates almost all the defects by trial and error. It takes a lot of computing time for each frame, but it cleans almost everything.



I try to give the right expectations to my customers. Difficult repairs can be more expensive and yet achieve only bad video quality. That’s a tough business.

]]> http://www.aeroquartet.com/wordpress/2008/08/13/expectations/feed/ http://www.aeroquartet.com/wordpress/2008/08/08/movie-repair-service-year-two/ http://www.aeroquartet.com/wordpress/2008/08/08/movie-repair-service-year-two/#comments Thu, 07 Aug 2008 22:09:39 +0000 Benoît Joossen http://www.aeroquartet.com/wordpress/2008/08/08/movie-repair-service-year-two/ After one year of activity, and over one hundred movies repaired, I can start drawing the first conclusions about this incipient business.

I’m amazed by the diversity of movies that have revived through my hands:

A submarine hockey game, an award delivery in name of the Queen of England, 2 house robberies caught on a spy cam, a few indie concerts, a dozen family events, from births to … burial, a dentistry congress, a Chernobyl nuclear plant tour, funny classroom stuff, a few documentaries and even a feature-length movie.



Diversity also in terms of codecs and formats:

Video: DVCHD Pro, DV, JPEG, RPZA, MPEG-4 Video, H264, Sorenson 3, Uncompressed, HDV, Apple Intermediate codec, ProRes422, XDCAM EX Pro.
Audio: AAC, Integer of all sort, uLaw

My customers are in general video professionals, but there’s also a number of hobbyists that value their lost movies enough to afford a repair. The two main causes of corruption are: technical problems during recording, and storage failure (hard disk, memory card).

First of all, let’s take a look at it from the problem solving prospective.
It’s definitively a problem that is useful and interesting to solve, thus meeting my top goals as an entrepreneur.
But, it’s not an easy business: it’s time consuming, needs a lot of interaction with customers, and doesn’t make a lot of money at the end.
So is it time to give up and move on? Well, not so fast. As any incipient business, it has to go through several phases.

The first months, it started as an experiment. At that moment, I did not really care about making money (but charging for the service was a reality check that you must put in place soon), nor about being able to repair any movie a customer would throw at me. The goal was to check that movie repair delivered as a service was possible. In other words, that the mechanics of the repair that I had envisioned, from the first contact with the customer, the transfer of data, diagnostic, delivery of a “repair kit”, iterations to achieve full satisfaction, until the payment, was working. With the first dozen of happy customers behind, the goal was considered achieved.



Once you know it’s possible, you want it to be reproducible, reliable and predictable: How will I advertise my repair service if I don’t know what can or cannot be repaired, what level of quality to expect? That’s what the second phase was about, and it took me the rest of the first year to fully characterize this.
With over one hundred of cases solved, I’ve developed all the tools and techniques, classified the cases from easy to impossible, the quality of repair from rough to broadcast quality: At the end I was able to repair almost anything, and to predict the quality that I would achieve.
This second phase required some sustained effort, so I wanted it to be profitable. I adjusted the price points accordingly.

Now it’s time to enter into the third phase. It’s about scaling up: to reach more customers, and increase productivity. I have a few ideas about how to reach them (more about this in a future post). I will continue standardizing and automating the repair techniques to be more productive. Ideally, I would end up using fully automated repair tools that could even be deployed as web services. I think that it’s possible in a few happy cases, and that for the rest I will continue having a mix of manual and automated processes. More productivity means also profitable at cheaper price points, thus potentially reaching a lot more customers.

All this is really interesting for me. I have serious technical and marketing challenges ahead, and this business can succeed if I shape it as a small industry. Each industry has its own rules, that reflect the balance between what the customers want and are willing to pay, and what technology can do.

I will succeed if I can discover those rules. I will fail if I play against them.

]]> http://www.aeroquartet.com/wordpress/2008/08/08/movie-repair-service-year-two/feed/