Project Complexity and Memory Needs

The 'Windows Movie Maker cannot save the movie to the specified location' error message is all too common. It tells you to verify your source files are available, the saving location is there, and there's enough free disk space. You have all of those things, and it still can't save. We've learned through experience that the message isn't complete enough.

It's become routine to advise users that their project might be too complex, and to suggest breaking it down into parts, render the pieces, and then put them together into the final movie... and most of the time it works. One of my fee-based services is to take the complex project, do the project splitting, and return the sub-projects ready for rendering... I've never had that work-around fail because the issue was something else.

Why does breaking it down and rendering the parts work? Because there's a direct relationship between the complexity of a project and the amount of available memory. You can easily check the status of your memory, and watch it during the rendering to see how close it gets to the limit... even watch as the rendering crashes when memory usage reaches the limit, followed by the error message a couple seconds later.   

My Toshiba was in the shop for repairs... from August 1 to Sept 6... and I was working on my older less powerful Dell laptop. With only 144 MB RAM and less than a GB of available hard drive space, I took the opportunity to check project complexity against available memory. I'm still looking for the easy to use meter that tells me how close I am to the limit, before having to find out by doing the rendering. Lacking that, I'm trying to develop rules of thumb about project complexity. I'm not there yet, but this newsletter gets me a step closer.

... before getting into it, here are a couple notes...

Notes...

My Toshiba laptop returned on Tuesday, with a new DVD/CD combo drive and TV Tuner. Both of them were replaced on the last trip just 3 months ago, and the TV Tuner had never worked since then. CompUSA told me they needed to reinstall the operating system from scratch if I wanted to see TV on the new tuner.

When it left this time, I told them to do what they needed for me to see the TV. It came back Tuesday with the operating system exactly where it was when I purchased the laptop in May 2003. It was working well and I could see TV, but I had more than a few software installations and updates to do.

The local CompUSA filed two requests to replace the laptop with another unit because of repeated issues... it wasn't approved. Maybe the 3rd request will be. It's 3 days since getting it back and my 'standard' issues started again tonight as I finished this newsletter.

.... on to the main topic

a Test Project

I started with an existing wmv file... the 'XML the Beginning' tutorial made recently by the Persian Poet Gal... a 2 minute and 10 second, 640x480, 602 Kbps,  7.6 MB file.

I could have started with anything... the goal was simply to make a project more and more complex, render it to a movie at each step, and see how much memory was needed.

Over and over, add more complexity after each successful render, assess the extra memory needed, and render again until I ran into the upper limit and got the error message.

the computer system

The Dell laptop has a 266 Mhz CPU, a 4 GB hard drive, and 144 MB of RAM. I bought it new in June 1998 with Windows 98.

In August 2004 I upgraded it to the Home edition of Windows XP.

I had no problems doing the upgrade and the laptop has been running the best it ever has... even though it's far short of the system considered minimal for XP and Movie Maker 2.

Starting the Project

Before opening Movie Maker, the Dell was using 116 MB of the 144 MB (80 percent of the installed RAM)... it wasn't going to take much complexity to use of the rest of the RAM and start digging into virtual memory...

Memory usage varies... I assessed things as I went using the Performance tab of the Windows Task Manager. Before starting a Movie Maker project, there are memory demands on the computer due to the operating system and other software running.

Opening Movie Maker 2.1 added 8 MB of memory to the total used... not much. The Windows Messenger service uses over 20 MB when I'm not even online. Sonic's MyDVD 4.5 uses 58 MB of memory when opened, before starting a DVD project.

Another 24 MB of memory was needed to create the start of the project , to get it to the point shown at the left... the single clip of the starting wmv file was on the timeline and split into 5 sub-clips.

Total memory usage at this point was 148 MB, which was 4 MB over the installed RAM... the project was starting to use virtual memory. When it does that, there's a significant slowdown as using the hard drive is much slower than using RAM.

If you don't mind going that slow, you can keep going until the combined total of RAM and virtual memory reaches your established limit.

Virtual Memory

The virtual memory setting gave Windows XP as much as 432 MB of the hard drive space to use as if it was installed RAM... adding that to the 144 MB of RAM, the Dell could go as high as 576 MB of memory before it gets the message from Movie Maker about not being able to save the movie.

On this system, most or all of the memory available for rendering is from virtual memory, not installed RAM.

To change the setting, use Start > Control Panel > Performance and Maintenance > System > Advanced Tab > Performance Settings > Advanced > Virtual Memory > Change...

If we run out of memory, we can increase the virtual memory maximum size instead of purchasing more RAM... right?? Yes, we'll see.

Make a mental note of the Initial size of 432 MB... we'll mention it below.

Add Complexity and Watch the Effects on Memory Usage...

Anything added to a project adds complexity... and more complexity adds more demands on memory. How much?? We'll look.

Adding complexity happens routinely as you work on the project... keeping an eye on the effects of the complexity, and where you are relative to your memory limit by using your Task Manager...

Task Manager - Processes Tab

The Processes tab of the Task Manager helps you see what's using your memory, and lets you close down some processes that are not needed when you render a movie... that's one way to free up some additional memory if needed.

Movie Maker, with no project open, is using a bit over 12 MB of memory... the two lines starting with NAVA... are for Norton Antivirus, things I could close down to gain 6-1/2 MB of memory. I tend to leave them and other processes running.

Task Manager - Performance Tab

To track the effect of adding complexity, use the Performance tab of the Task Manager.

The Commit Charge (Memory Limit) is the key item to watch. Compare it to the other two numbers in the lower left block, the Total memory being used, and the Peak memory level reached since the computer was last started up.

This picture shows the laptop after a fresh reboot, with Movie Maker running but no project open. 

It's telling me that the current memory demands from all running processes total 142 MB... that the current Limit is 356 MB... and that the highest or Peak demand for memory since the laptop last started up was 143 MB.

The 432 MB Initial Limit for Virtual Memory, noted above, shows up here in the 356 MB Limit (144 MB of RAM + 216 Initial Swap File size = 360 MB).

After checking the running processes and the memory needed for them, and setting up the Task Manager to watch things... it's time to add the complexity and note the effects. In doing it, realize that these numbers are precise, but things change so dynamically on the system that you can only use the numbers as approximations when trying to draw conclusions.

Rendering the Simple Project

To see how memory needs change with the choice of quality, I did some renderings without changing the project.

The memory used just before rendering was 148 MB. The memory needed to ramp up to the first couple percent progress points (I didn't spend the time to finish the renderings... the memory needed ramps up pretty quickly to the maximum level for the profile selected):

• 1 MB limit - 170MB peak memory needed - very steady memory usage thru the rendering - it fell instantly to 155MB when rendering finished
• 2 MB limit - went up to 181MB within a percent or two... and leveled... cancelled - fell to 160MB
• 4 MB limit - went up to 194MB within 2 percent and leveled... cancelled - fell to 149MB
• 8 MB limit - went up to 189MB within 2 percent and leveled... cancelled - fell to 152MB
• 16 MB limit - went up to 251MB within 2 percent and leveled... cancelled - fell to 149MB
• 32 MB limit (the max for the size setting was 33 MB) - went up to 268MB within 2 percent and leveled... cancelled - fell to 149MB
• DV-AVI - went up to 200MB within 2 percent and leveled... cancelled - fell to 149MB

Here's a  chart to show these results... quality (file size) versus peak memory needed to render it.

The horizontal red line shows 46 MB was needed to render it to a DV-AVI file... the numbers in the vertical bars are the file sizes of the compressed WMV files, with the scale at the right showing the memory used to render them.

Maybe I made a mistake noting the memory needed for the 8 MB file... or maybe it really was an anomaly... it doesn't change the overall pattern so I didn't do a retest.

The bottom line is that, for a given project, rendering higher quality compressed WMV files needs more memory than rendering a DV-AVI file... and rendering a low quality compressed file needs less.

With the highest memory demands being for a high quality WMV file, I did all the remaining tests using the WMV option at the highest file size choice that Movie Maker offered... the maximum size allowed moved up as the project timeline got longer. As complexity increased, the size of the rendered movie went from the 1 MB limit above to one as large as 331 MB.

Adding Complexity

Here are the steps I used to make the project larger and more complex... and the effects on the memory needed. I looked at the changes in memory usage in two places. How much more was used when having the project open on the timeline, before the rendering? And then, how much more memory was needed to render the movie.

8 MB were used to open MM2, and another 24 to get the project started with its 5 clips. The rendering of the basic no-frills project to a 32 MB WMV file needed 120 MB. These are the baseline needs; I'll note the changes from there.

1 - added one random effect to each of the 5 clips... 14 MB more (60% increase) was needed for the project, but nothing extra needed to render it to a 32 MB movie.

This was the first clue that the addition of video effects has minimal effects on the memory needed for rendering.

2 - added a transition between each clip, 4 transitions total... with default durations... 24 MB more (60% increase) was needed for the project, and 17 more (15% increase) for the rendering to a 32 MB movie.

3 - maximized the transition durations by overlapping the clips as much as possible, reducing the total length of the movie. There was no change in the memory needed for the project, but 27 more was needed to render it, although the maximum file size choice had gone down from 32 MB to 20.

Steps 2 and 3 combined was the first clue that the memory needed to render transitions is more significant.

4 - added 5 more effects to the 1st, 3rd and 5th clips, to the maximum allowed of 6 per clip... the increased memory needed to work on the project was 9 MB and there was a decrease of 9 MB in the memory needed to render it to a 20 MB file.

5 - added 5 more effects to the 2nd and 4th clips... only 1 more MB was used when working on the project, and 12 more to render it to the 20 MB movie.

Steps 4 and 5 combined reinforced what I saw in step 1 about video effects not needing more memory to render the movie.

6 - rebooted the computer... which cleared memory to the tune of reducing the total used by 34 MB.

This was the first of a few reboots during the testing, each one showing significant memory reductions. Something was happening to clear the air during reboots.

7 - added a single music wma file... the sample Beethoven clip. 4 more MB was needed for the project itself and 15 additional for the rendering to a 20 MB movie.

Adding music added more memory demands than adding effects, but less than transitions.

8 - added a single text overlay to the first 40 seconds... using the scroll perspective animation. 14 more MB was used to work on the project and 48 more to render it to a 20 MB movie.

Adding a text overlay required significantly more memory, especially to the rendering step.

This completed the basic project... it was now a 1 minute, 20 second timeline with 5 video clips, packed with special effects and transitions, and topped with background music and a long text overlay.

At this point I rebooted and jogged the total memory usage back down by 45 MB... after the memory usage was cleared with the reboot, the project open in MM2 was using only 12 MB more than before I started adding the clips to create the project. The rebooting had saved over 65% of the memory being used as the project was being built.

This drove home what I had seen of the memory savings from the first reboot in step 6.

The renderings weren't close to running into my total available memory limit. I'll change tactics to increase the complexity from here.

Doubling and Doubling Again... till it Crashes

9 - copied and pasted all the clips in the project... doubling the project duration to 2:40. It used 39 more MB to work on the timeline, but memory to render it to a 41 MB movie went down by 20 MB.

I was wondering if copying/pasting the same clips results in some efficiencies over having totally different clips... I don't know. Even if there are efficiencies, the complexity was going up due to overall project size increases.

At this point, each doubling of the project also doubled the file size limit for the saved movie... the same profile was being used each time.

10 - doubled again to 5:21. Memory used to view the project went up another 19 MB, and memory needed to render it to an 82 MB movie rose another 13 MB... very modest increases in memory needs.

11 - doubled again to 10:41. Only 4 more MB needed to view the timeline, but an increase of 59 MB in what was used to render it to a 165 MB movie.

12 - doubled again to 21:23. 27 more MB to view the timeline... and didn't know yet what was needed to render it to a 331 MB movie file.

The project had reached the complexity level needed to two-block the memory of the Dell laptop. It had

• 16 repeating segments totaling:
• 80 video clips
• 6 haphazardly selected special video effects applied to each video clip - that's 480 total effects
• 64 special transitions, most of them significantly longer durations than the default
• 16 music clips
• 16 title overlays

The memory to do the rendering reached the limit of 576 MB (144MB RAM + 432 MB of virtual memory = 576 MB)...

I got the familiar error message... in this case I was happy to see it so I could get on with the rest of the newsletter.

Memory usage ramps up pretty quickly in the rendering process, reaching close to peak levels within the first couple percent of progress... that saves a lot of wasted rendering time, as you can cancel the process and think about what to do.

If memory usage is over 95% of the Limit, you might still reach it at some point later in the rendering. If you have more slack than that, you should be able to finish. 

13 - I was increasing project complexity to reach this point... but instead of breaking it back down into parts to render separately, I wanted to go forward by raising the virtual memory limit.

I changed it to a maximum of 500 MB...

... and rebooted...

... and tried the render again.

The Final Render

With the new higher virtual memory setting, I started the final render.

It started at 01:34 am on Sept 5th... and took almost 30 hours to complete. Obviously I didn't watch it the whole time. Here are some of the data points I noted:

493 MB of usage at 0% with a projection of 981 more minutes...
500 MB and still at 0% with 1317 minutes left...
529 MB at 1% with 1709 minutes to go...

529 MB at 2% with 1950 more minutes projected

... 3% - 1991 minutes left (the projected time was still growing to this point)

570 MB at 23% (with a peak of 583 MB so far) with 1540 minutes to go

541 MB at 24% with 1505 minutes left

612 MB at 54% with 900 minutes to go

622 MB at 60% with 790 minutes left

605 MB at 64% with 708 minutes to go (with a peak so far of 632,616)

590 MB at 69% with 597 minutes to go

... 10:30 am on Sept 6th - the rendering had finished sometime overnight. The peak memory usage was 641,548, just a few MB less than the higher Limit... I pressed the Finish button and watched the movie. It played fine...

Cross-Check on Rendering to DV-AVI

My Toshiba laptop had returned, and I moved the project over to it... I wanted to confirm that rendering this complex project to a DV-AVI file used less memory than the high quality WMV option I'd been using. The chart early on in the newsletter says it should be less memory intensive.

It confirmed it. Peak memory usage during rendering to a DV-AVI file was 359 MB on the Toshiba, while rendering the WMV used 400, over 10% more (it probably was higher... I cancelled the rendering when it was at a few percent complete).

Conclusions and Closing

The rendering on the Dell finished successfully, but it took almost 30 hours to render a complex high quality 21-1/2 minute movie... Rendering is CPU intensive and the 244 Mhz laptop was running mostly at 100% CPU utilization throughout... an advantage of it being a lower powered computer is that it didn't overheat.

What are the lessons learned or reinforced by this week's testing?

• Project complexity and memory demands go hand in hand... but so does the choice of the file type to render to (DV-AVI versus compressed WMV) and the compression quality level.
• On a relative rather than absolute scale, project complexity increases significantly more when special transitions and text overlays are added... compared to adding special video effects and music.
• Rebooting Movie Maker and your computer frees up lots of memory... do it just before the final big render and it could very well be the difference between success and the standard error message.
• If your project is too complex for your current virtual memory setting, change it. On the Dell with 144 MB of RAM, I bumped the Virtual Memory setting up over 15% from 432 to 500 MB... I could have raised it higher... Virtual memory is hard drive space... the 144 MB of RAM was less than 25% of the 644MB of total memory being used.

I look forward to comments and discussion about this and other newsletters on the forums at:

Windows Movie Makers.net

Have a great week...

PapaJohn

Movie Maker 2 and Photo Story 3 - www.papajohn.org
Photo Story 2 - www.photostory.papajohn.org

Products and Services

I'm involved in many things that support users of Movie Maker and Photo Story, and adding more regularly. Some are free and others reasonably priced.

Radio and Podcasting

TheDVShow is the only weekly Podcast having more useful information about desktop video editing and production than anywhere else on the Web. Digital video editing, nonlinear editing, streaming media, software releases, tutorials, business tips, technical help, download of the day and news on the latest products to make everything easier. It's where professional and consumer desktop video users go to stay on the cutting edge. 

Call the phone mail machine to get your technical question answered on the air... call (206)-203-3516

The radio broadcast is from Boston, and the website has downloadable podcast files. The June 19th 2005 podcast included the first 'bi-weekly' show with a segment about Movie Maker 2.

Books and Magazines

Movie Maker 2 - Do Amazing Things (with its online companion on www.papajohn.org), published by Microsoft Press

Movie Maker 2 - Zero to Hero - with support on the publisher's forum -  Friends of Ed

MaximumPC's winter 2005 quarterly special... had a 7 page tutorial 'Make a Killer Home Movie with Maker 2'. The special edition of the video made for it is now  on my website as a file download.

Learning VirtualDub - published by Packt Publishing, is the first book about VirtualDub software. I wrote the first chapter about downloading and setting up the software: VirtualDub, VDubMod and AVISynth.

Websites

Movie Maker 2 and Photo Story 3 - www.papajohn.org - the site's 3 goals are: an online companion to the Do Amazing Things book, a detailed tutorial for PhotoStory 3, and helping you solve Movie Maker 2 problems.

Online Support - Forums and Newsgroups

I'm a regular on many online forums and newsgroups, the key ones being:

Forums are open to all for viewing, but require registration of those who want to post. Moderators actively participate to ensure the forum discussions move forward and stay on track.

Movie Maker and Photo Story forums at W indows Movie Makers

Movie Maker 2 forum at SimplyDV.com

Newsgroups are wide open for all to view and post... moderation is collective by the participants.

Windows XP Movie Maker newsgroup - microsoft.public.windowsxp.moviemaker

Photo Story 3 newsgroup -  microsoft.public.windowsxp.photos

Weekly Newsletters

Movie Maker 2/Photo Story newsletter. The annual subscription is $20 and the link to subscribe is on the main page of my Movie Maker website at:

www.papajohn.org

Topics for upcoming newsletters (always subject to change):

Older newsletters (more than 6 issues ago) are posted by Rob Morris to an Archive Site at his Windows Movie Makers website. Links from website pages to specific newsletters make it easier for the website viewer to see the content of both while browsing a topic.

Drop an email to suggest a newsletter topic... I can use more requests rather than fewer.

Software

Add-On Transitions and Effects

Transition Maker 2  (TM2) is a utility for the ultimate in making your own personal and custom transitions for Movie Maker 2. It's a joint product from Patrick Leabo, the programmer, and myself. Version 2 was released a week ago and I'm still working on updating the online tutorial.

I routinely beta test the  Pixelan packages and think very highly of their people and products.

ProDAD's Adorage package for Movie Maker 2 provides an additional source of professionally developed transitions and effects.

Personal Database

Managing your personal information is more of a challenge as hard drives get bigger and the internet more robust. 

My personal database has been an ongoing project over many years, and is now available to others. Info is on the Managing > Personal Database page of my site, and in the database package itself.

It's available free to regular newsletter subscribers... send an email request.

To others it's $10. To order, use the button on the top of the Managing > Personal Database page.

Online Gallery

An online gallery that fully aligns with the main priority of the website is the  'PapaJohn Expert Zone' at neptune.

Check it at Neptune  and the Distributing > Neptune page of the website, where there's a developing tutorial about how to use the service.

Training

in conjunction with the Portage, Michigan library, I offer two free training sessions about Movie Maker, an intro session and a workshop. The upcoming scheduled sessions are:

Monday - September 19 - 6-7:30 - Intro to Movie Maker

Other fee-based services

If you can't save a movie because your project has become too complex, e-mail a copy and I'll divide it into manageable sub-projects, and provide detailed instructions about how to render the parts and assemble them into your final movie. $49.95 (no cost if it's not the right solution or doesn't work) - for details, see the sidebar on the Problem Solving > Can't Save a Movie page of www.papajohn.org

Movie Maker 2/Photo Story training and support services start at $50 per hour - send an email -  PapaJohn@CharterMi.net  and I'll help you determine your needs, and work with you to plan and implement them.

Wedding combo website/video packages - starting at $2,500 + travel expenses. See  Jill-MarkWedding or the bottom branch of the Movie Maker 2 website for a sample of what you can expect for the online portion of the package.

Microsoft is a registered trademark of Microsoft Corporation in the United States and/or other countries.

About John 'PapaJohn' Buechler from Microsoft.com John "PapaJohn" Buechler, of Kalamazoo, Mich., goes by PapaJohn online. An avid user of Movie Maker since its first release, and a regular supporter of the community of Movie Maker users, John received a 2003 MVP award from Microsoft for that support. In March 2003, he started a comprehensive website about Movie Maker 2 at www.PapaJohn.org. He maintains the website, writes books and articles, teaches, and provides support services - all for the community of Movie Maker 2 users. An engineer by formal education, John is a computer database and multimedia expert by business and personal experience. He co-authored the first book about Movie Maker 2 and is actively working on a second one. You can find his advice in the Windows XP Movie Maker newsgroup and in the Windows Movie Makers Forums. This newsletter is republished with permission of John "PapaJohn" Buechler. Please note that this is an archive of newsletters and some information may become outdated. PapaJohn, and the webmaster of this site, provides this information "AS IS" with no warranties. Visit - PapaJohn's Movie Maker 2 and Photo Story 2 Newsletter Index