Yesterdays problems, is todays success.

After finishing my blog yesterday, I decided  to use another technique to solve my problem.

So I know the read speed was not good, I wanted to isolate the problem, or try to exclude a problem by testing, individual components.

So I wrote a small test program for libdvdcss, by doing this I hit nail on the head, this was where speed problem was.

First run it showed about 64kb/s a bit more then when I was benchmarking from mplayer.

Corrected a few mistakes and I was at 200kb/s.

The experimented with my read ahead cache implantation, and I was soon at 800kb/s.

With bit more work I managed to get it up to 24000 kb/s, this is really close to the copy speed I got yesterday from DVD to HD.

So I think we can say the speed problem is solved.

The problem with mplayer is that it tries to read one by one block, but CD has spin up, it has to find the sector where block is located, and this for every time you read one block, this is not efficient.

sb600sata.device or sata device driver for X1000, does not try to pre buffer, this is the problem. So this is way I need to do that in mplayer/libdvdcss.

So what I try to do is, avoid any one block reads by pre fetching blocks, as mplayer mostly reads blocks in sequential order this pretty easy to do, so next reads is going to be really quick, until the read offset is outside the buffer and it has to fill the pre fetch buffer.

Also because its in memory, I do not need to foreword the IO read to the DVD read process I created, it can simply just copy the data from the buffer, so no need to wait for task switch, signals and stuff like that.

I call my pre buffering stuff "Evil" because it was quickly proto typed, as future note it should be moved into the dvdcss struct, so etch dvd device can have its own pre fetch buffer.

Anyway its not a problem for mplayer as I think there is only one process reading at any given time, from one device, I'm shore this might not be case for other operating systems.

Way I think this is not problem is because libdvdcss is not a shared library, the the pre fetch buffer is pretty mush private to mplayer as its static linked.

The mystery of the slow DVD speed and Mplayer for AmigaOS4.

I have been investigating the issue, for some time.

Trying to find out of problem was caused by read errors, and trying to locate the problem.

What I have done is fix cache and device io in Mplayer as it was broken, I was hoping by doing this the problem was going to go away.

But what it did was, it enabled mplayer to pre buffer 999999 bytes, but as soon as the buffer runs out mplayer is back to crawl speed.

I have tried to compile the latest version of mplayer and ffmpeg, and the problem remains. I have tried to read from the filesystem instead of depending on the IO system, by fixing that part of the code as it was broken as well, but there is no luck there, reading a DVD is not like reading a file.

I measured the read speed, at device IO in libdvdcss/device.c and found the 38KB/s read speed, so in other words, the read speed is low, it should be around 1.49MB/s, so we are not even close, to be playable.

I made some tests to see what best copy speed was, I copied a 972Mb file in 372 sec, or 2.6Mb/s, so this is not a hardware problem, the problem most be with DVD://1 DVDNAV://1 -dvd-device option.

I can play the a VOB file from a DVD disc, with some cache, but only this way I get good speed, with out cache its just as slow as the DVD://1 DVDNAV://1 options.

I have taken the time to look at the status line in mplayer, it says some thing like:

A: 13.1 B: 3.2 A-V: 9.981 ct: 0.051 77/155 39% 24% 10192.1% 153 0

So what this status line says is that A: is for audio, it represent the time played for audio, and V: for video, A-V: is the out of sync number it should be 0, 77/155 is decoded vs the not yet decoded frames in buffer, 29% is codec, 24% is the video, and 10192.1% is the audio,

the the biggest problem looks to be related to audio, but its not really, the high number comes because the audio buffer is empty and the audio cache has to be filled, so I tried to trace the it up stream, from main loop in mplayer.c, but it was not easy.

So then I remembered about debug features in AmigaOS4.1, so I decided to collect some information using stacktrace feature.

There was a problem it required the task I wanted to be halted, while using back trace, but that was lucky not a issue for me, as device io code was moved into a different task of its own, to prevent problems with treading in mplayer, and task that issues the read waits for signal for data to be ready, so I was happy about that, if not might not have tried this.

Anyway here is stack traces I made, using Exec debug interface.

Stack:

kernel : NULL

kernel : NULL

mplayer : dvdcss_seek

mplayer : css_seek

mplayer : UDFReadBlocksRaw

mplayer : DVDReadBytes

mplayer : ifoRead_PGC

mplayer : ifoRead_PGCIT_internal

mplayer : ifoRead_PGCI_UT

mplayer : ifoOpen

mplayer : open_s

mplayer : open_stream_full

mplayer : open_stream

mplayer : main

newlib.library.kmod : NULL

newlib.library.kmod : NULL

newlib.library.kmod : NULL

mplayer : _start

dos.library.kmod : NULL

kernel : NULL

kernel : NULL

Stack:

kernel : NULL

kernel : NULL

mplayer : dvdcss_seek

mplayer : css_seek

mplayer : UDFReadBlocksRaw

mplayer : fill_buffer

mplayer : stream_read_internal

mplayer : stream_fill_buffer

mplayer : cache_stream_fill_buffer

mplayer : demux_mpg_fill_buffer

mplayer : ds_fill_buffer

mplayer : ds_get_packet_pts

mplayer : decode_audio

mplayer : mp_decode_audio

mplayer : main

newlib.library.kmod : NULL

newlib.library.kmod : NULL

newlib.library.kmod : NULL

mplayer : _start

dos.library.kmod : NULL

kernel : NULL

kernel : NULL

Stack:

kernel : NULL

kernel : NULL

mplayer : dvdcss_read

mplayer : css_read

mplayer : UDFReadBlocksRaw

mplayer : fill_buffer

mplayer : stream_read_internal

mplayer : stream_fill_buffer

mplayer : cache_stream_fill_buffer

mplayer : demux_mpg_fill_buffer

mplayer : ds_fill_buffer

mplayer : ds_get_packet_pts

mplayer : decode_audio

mplayer : mp_decode_audio

mplayer : main

newlib.library.kmod : NULL

newlib.library.kmod : NULL

newlib.library.kmod : NULL

mplayer : _start

dos.library.kmod : NULL

kernel : NULL

kernel : NULL

So I hope this information is useful to find out where the problem comes from, if not it gives a good overview of where problem might be located I hope.

The problem with mplayer is to get a overview, anyway if anyone knows about free vitalization tool, to get some overview of this source code it might be rally useful.

http://cs.smith.edu/dftwiki/index.php/Visualization:_Old_Blog

AmigaONE-X1000 HDReady...

AmigaONE-X1000 HDReady...

I have been thinking about writing some thing about the fastest Mplayer from me, and about latest utf8.library, I guess I should say something.

Your AmigaONE-X1000 is now HDReady, you can now play 720p video whit out problem, thanks to a changes in vo_cgx_wpa driver, and the new vo_comp, the problem whit old vo_cgx_wpa driver is that it uses RGB, and not ARGB, the Radeon HD cards needs ARGB, so the result was that graphic needed to be converted before being sent over.

The second problem was that it was using software scaling, this slow and CPU intensive, by using hardware acceleration to do the scaling we can scale windows to full screen and back, whit using all the CPU power.

And I wanted to compile a fully AltiVec enabled version of MPlayer. The vo_cgx_wpa and vo_comp is also useful for Sam460 users, it does help to reduce CPU load. How it all came about:

Back story:

well it started because I realized Ka1e needed help, whit MUI-MPlayer, he was begging Hans to emulate overlay, Hans did not want to spend time on obsolete technology. Nor did he have time for that, so this is way I did take on this challenge.

History for Mplayer for AmigaOS4.

The MPlayer project for AmigaOS4 started whit me I think, at least for AmigaOS4, the MorphOS version already existed before I tried to compile it. more importantly my attempt showed it was possible to compile it.

Click this link to read how it all started

I did not manage to compile it because some fork() stuff, but Andrea Palmate'(‎'afxgroup'), did and so it became his project, Varthall and Abalabanb started a project a project on code.google.com.

Now I'm going to tell you some thing not every one knows, or at least where few. After Andrea Palmate successful ported MPlayer to AmigaOS4, I requested the source code from him, and worked on a plugin for IBrowse 2.4, but due to complications and way I decided to do it failed, this project ended in yuv420 to ARGB routine and some stuff, that project is dead now.

http://www.amigasoft.net/pages/mplayer/mplayer.asp

While I never got MPlayer IB2.4 plugin to work as good as I hoped for the work on IB2.4 plugin did end up in plugin for Audio, this probably the route I take if where to redo it.

http://www.os4depot.net/index.php?function=showfile&file=network/browser/np_audio.lha

So I'm well familiar with MPlayer well before I started on this projected. While I was not one to implement MPlayer I was the one who initiated it on AmigaOS4, so it bit funny to see whit project coming back to me, or at least partly.

Today this is no longer needed for a IBrowse plugin whit HTML5 the web browser does no longer need a plugin to play videos, thanks to the Excellent work of Fab and Ka1e.

What is future for me and Mplayer, I have not yet decided, I have some ideas, more likely its going to be fix up mplayer-amigaos at code.google.com at least. Now that Me and Ka1e has joined the code.google.com mplayer-amigaos effort, I like to see is FAB and Andrea Palmate joining the mplayer-amigaos code.google.com effort, so this becomes team work, so whit don't end up whit different MPlayer whit bug that's already fixed in different MPlayers and so. Even if all mplayer can be based on common code base, it might be possible to have unique versions of MPlayer I believe.

https://code.google.com/p/mplayer-amigaos

Its always good to experiment.

While things does not always work out some time you learn something that can be of use latter, this is a example of this at lest for me. I can also take the Basilisk II as an Example, Basilisk was first please I used Composition for scaling window content, part of that code made into Mplayer vo_comp. I have also been using Composition in Excalibur, to do alpha blending also.

UTF8.library, progress and some information about the up coming version 4.

I'm a bit amazed by number of download of my UTF8.library, for library that is not used by any application it gets a lot of downloads, it does make my happy.

47 Downloads currently on version 3, not bad at all,

A new version will be out in today or tomorrow, I think, just need to check that every thing is ok.

The new library will contain 7 more string handling rutins. 1 more routine to check Byte Size.

As you might have noticed, naming of function I use Relefact names used in std:string C++ class function names Microsoft uses on there string class. Of course there are some variation because we need to support for sheard memory, when dealing whit strings.

As I'm nearing completion, I hope that I can return to the project I was working before starting on UTF8.Library.

There might be another 10 to 15 routines we don't have yet in this library to render the other libraries redundant. While your waiting you just need to mix and match what you need for that different libraries.

I will so like some feed back form you developers on what routines you like to see in the next version, what do you think I have forgotten, are there thing you think I should do different?

Better to tell me now before I decide to freeze features.

Also the language alphabet has been a bit annoying, I hope its possible to get this from some where in the OS, or else I will need to ask people provide there Language alphabets in UTF8 format as text file.

Because we will need it do casecmp, upper, lower and sorting, the standard one's only really support English it seams, and thats no good.

Things I'm working on for the mommet.

I have been busy reading about MS C#, and about classes, in order to make it interesting I wanted to make a program, that will be useful to me and maybe others, a cross platform application.

I know C# is not really portable but its more portable then VirtualBasic, at lest it has most of the C++ syntax, even if .Net Framework is different, there are a few different versions of .Net Framework you have XNA a Game API used on XBOX, and then Portable Class Library.

The Portable Class Library is more or less useless, because everything has to be recreated anyway, and its too small to be of any use.

Supporting just XNA does not make lot of senses, if AmigaOS can support some thing, there is no point in limiting it to XNA.

The full .Net Framework is too big of job to support, so what I have basically been doing to create C++ classes for what I have used in my project.

And been trying to get same program to work on AmigaOS whit small modifications, just by implementing the classes need to for the program.

So to make this clear this is not XNA, .NET or Portable Class Library this is some thing new.

Hopefully this will open some door for cross platform applications and portability between the two operating systems.

The only problem is that AmigaOS has yet to come out of the stone age.

Like File Notifications is way better on Windows, then on AmigaOS where its pretty primitive.

UTF8 encoding really sucked on AmigaOS, but works well on Windows, and I believe it has become the standard encoding for text files in windows7.

So I have spent a week trying to figure out how to do things, and ended up whit writing my own UTF8.Library it has now been uploaded to OS4Depot.net.

I'm hopeful that UTF8.Library will make more programs UTF8 friendly. 

 

More information to come on what I'm working on.

So here comes a small blog update.

Well I promised to fix few issues on forums, they where fixed, lately I have been feeling where tiered, and too many things to think about, during xmas, I have not felt motivated to work on it a lot, or get it completed before xmas, but now its here.

Every software release takes some time to prepare, just making shore the changes logs are updated, and checking that things works, for this release, I needed to check if it worked on my old AmigaONE-XE as well as AmigaONE-X1000, also when I'm finally ready, I find small things that need to be fixed, like some of the icons where not PNG, and also, I have noticed I have edited some files I should have generated, not really fixed that one.

The problem for language support you need to edit a source file, this will generate the local.c and local.h files, so in other words, I think local language support is a bit messed up, I should investigate how this works and make shore the files are generated correct, I will see what I can do for the next version.

Also I was thinking about making a YouTube video of fix, and what I did, but its way too much work for too little.

Its better if just explain what the problem was, and how I fixed it.

Fist of all there was some yellow effect that we did not like in full screen mode on AmigaONE-XE, Pagasus-II and other AGP/PCI AmigaOS4 compatible computers, Sam460 and AmigaONE-X1000, did not have that issue, so I need a fix that worked fine for PCIe as well as PCI/AGP.

I fingered out that the lack of picasso96 lock was the problem, but I'm not a fan of p96 lock if I do not need it, as p96lock has a tendency to lock up graphic system for a period of time, so I made it a option.

Last time I released Basilisk II 1.0.3. I did not write a blog but record a video, the volume was recorded a bit low, you might need to turn up the volume.

Her is the link:

And also thank you eliyahu for your donation, your the first :-)

Its time to talk about optimizing code, so I was looking at Basilisk II code.

Its time to talk about optimizing code, so I was looking at Basilisk II code.

One think I noticed was was I did have his platform depended read and write functions.

So I found where they where used, this where used whit UAE CPU memory header by read and write functions for CPU core, and some where else, this where yet agin used some where else, by some other function.

As it happens my education was electronics and programing on microprocessors, it was impotent back then to reduce code due to slow CPU & MCU, micro controllers and micro processor units.

We used to use inline assembler and macros. So now I'm going to show you the magic power of macros, and how it can make big difference.

So this code show what was happening in Basilisk II, you have two loops typical when you filling or doing some graphic operation, so you have a main routine calling do_sum and do_sum calling sum, but this can be changed, this is how it works, so what do you do macros.

Now lets see what difference it made.

So we have 4096 ms (4 sec) before the change.

after the change we have 1091 ms (1 sec) 

so what we find is that 4096/1091 = 3.754. so the unoptimized code is 3.7 times slower or 375% slower.

so what is going on?

functions generate a lot of code, and they forces program to make jumps, jumps are considered expensive, often result in cache misses, next functions are designed to be isolated, variables from the previous code put on the stack to be later restored when you exit the function, function parameters are filled, to be letter read, by code inside the function, and when its all done it old variables are restored and the program exits the function.

So as you can understand its a lot of stuff that goes on when calling functions.

now when we rewrote it to use macros, the the macro is just replaced in the code before its compiled, the result is that that we only need to increment a variable by a value, and this in the main function.