[01:13:12] *** Quits: benedikt93 (~benedikt9@unaffiliated/benedikt93) (Quit: Bye ;)) [02:03:55] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [02:03:55] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [03:45:42] <[7]> user890104: hm, the stats are getting worse [05:50:43] [7]: can they be caused by turning the ipod face down, because the lcd backlight is shining too much? :) [05:51:10] <[7]> wouldn't be impossible [05:51:22] <[7]> the orientation of the drive can influence the behavior [05:51:33] <[7]> however it's good to catch all the damage now rather than later [05:53:54] hm, some of them seem to be zeroed at some point [05:54:16] [13:25:19] write suspect: 2, write suspect lockup: 0, write slow: 0, write error: 2, write lockup: 0, read suspect: 254, read slow: 0, read error: 81, read lockup: 0, read data error: 0, read slow data error: 0 [05:54:33] for example, read/write suspect vs. now [05:55:17] <[7]> yes [05:55:34] <[7]> "suspect" means "something in that area went wrong, rescan in more detail" [05:55:50] ah, i see [05:55:51] <[7]> they'll move to either OK or one of the error states shortly after [05:56:33] <[7]> in fact it will only continue to the next 32MB chunk once all non-known-bad sectors in the chunk can be written and read back successfully 5 times in a row [05:56:46] <[7]> (taking care of wiping the drive's cache in between etc) [05:57:12] <[7]> figuring out how to get around the nasty caching behavior and firmware bugs of these drives was the most complex part of it :) [05:57:57] and why is write ok so small? isn't it supposed to be almost equal to read ok? [06:04:11] <[7]> hm, or looks like the chunks are 64MB, not 32 [06:04:19] <[7]> read ok implies write ok [06:04:30] <[7]> write ok means "was written, needs to be read back and compared" [06:04:43] <[7]> read ok means "was read back and data matched expected value" [06:05:29] * [7] re-runs hddscan2 on his classic after it was sitting on his desk for years [06:05:38] <[7]> let's see how much the damage has grown since then... [06:06:06] <[7]> first 16K sectors chunk had like 10K write suspect [06:07:11] <[7]> it managed to slowly chew through them with just one write error, but I bet there will be a whole bunch of read errors [06:08:34] <[7]> 99 read errors within the first 16K sectors during the first pass alone... this one is MUCH worse than yours! [06:08:56] <[7]> but it seems to be stable, second pass didn't find further problems [06:10:33] <[7]> interesting [06:10:40] <[7]> this tool is more clever than I even remembered [06:10:58] <[7]> looks like it works one 64MB chunks, but only moves 32MB steps forward [06:11:10] <[7]> i.e. chunks overlap by 50% [06:11:22] <[7]> so each sector is actually checked 10 times in different situations [06:11:54] <[7]> (or even more if ANY other errors in that chunk did turn up during a repeated scan) [06:12:44] <[7]> first hard reset... [06:13:09] <[7]> (i.e. power cycle because the drive completely locked up, refusing to soft reset within a minute) [06:14:26] ah, i see. yours is really worse than mine [06:14:47] <[7]> first bad sector is sector number 37 [06:15:05] <[7]> (yes, I received the ipod in that state ;) [06:15:21] if you're going to run it until the end, can you set up my screen mirroring script? [06:15:49] <[7]> depends on how easily I can get that to run :) [06:17:22] well, a shell script (in your case, .bat in mine) with an infinite loop, doing emcore.py downloadfile from 0x9C0D068 with size 0x38400 and sleep 5 [06:17:48] <[7]> the address will be different on mine :) [06:17:57] and a php script, that is pointed to the output of emcore.py, that needs gd2 and access to the file [06:18:10] and a r805 build of mallocdump [06:18:32] <[7]> I'd probably do a full memory snapshot and run tlsfanalyze of it :) [06:18:57] well, if you prefer :) [06:19:17] <[7]> hm... maybe I'll reimplement that in a more sane fashion tomorrow [06:19:28] i didn't want to interrupt the process [06:19:58] *** Quits: [7] (~quassel@rockbox/developer/TheSeven) (Disconnected by services) [06:20:07] *** Joins: TheSeven (~quassel@rockbox/developer/TheSeven) [06:21:23] and the php script: http://pastie.org/pastes/8252138/text [06:23:18] tlsfanalyze basically analyzes a full memory dump and displays the allocations [06:23:20] some kind of offline dumpmalloc [06:23:24] (which is far more fobust in case of corruption) [06:23:29] robust* [06:23:35] it also pinpoints the corruption, if there is any, that's what I originally wrote it for [06:23:40] woah [06:23:44] that looks rather inefficient :P [06:24:49] it was even worse [06:25:14] i was parsing the contents, while reading the file [06:25:29] which caused a lot of race conditions [06:26:17] http://pastie.org/pastes/8252146/text the loop should be a bit faster now [06:27:35] nah, shouldn't make any difference at all [06:28:16] I'd suspect that the ord($file[$pos++]) version is very slightly faster [06:28:32] and the race condition is somewhere else [06:29:13] it was there, when i was using read($fp, 3) inside the inner loop :) [06:29:20] fread* [06:29:50] well that's even worse of course :) [06:29:59] the other race condition is on the ipod, if i read a half-written framebuffer [06:30:24] yes, I've hit that a few times [06:31:27] http://paste.pm/99l.c [06:32:00] so I guess 092F45C0 is the framebuffer... [06:32:35] uhm, this is what it looks like [06:33:40] isn't it supposed to be owned by the kernel? you are writing to the console after all [06:35:18] nope [06:35:43] hddscan2 is drawing this itself and sending it to the kernel as a bitmap [06:35:52] otherwise you wouldn't see a framebuffer allocation at all [06:36:18] the emcore console isn't clever enough to support this kind of application reasonably well [06:36:40] UMSboot's new one is better, that's a full ANSI-compatible console [06:38:33] so you can write e.g. "this is ^[[31mred^[[0m text" to it [06:39:13] (with ^[ representing ascii 0x1b [06:40:24] ah, i have to try this :) [06:42:40] user890104: http://websvn.freemyipod.org/filedetails.php?repname=freemyipod&path=%2Fumsboot%2Fsrc%2Flib%2Ffbconsole%2Ffbconsole.c of you're curious how it's implemented [06:43:15] (yes, it has pluggable framebuffer and text renderer modules) [06:44:06] ah, i didn't know about this syntax: case 30 ... 37: [06:47:24] looks like it can be ported to emcore [06:47:44] or maybe porting emcore to use the whole "framework"? [06:49:36] I'd be more in favor of the latter, but there are a few problems [06:49:47] first it's an awful lot of work of course [06:49:58] second: we should add some more abstraction while we're at it [06:50:12] third: the current framework isn't thread safe at all [06:52:48] ~1.3% damaged sectors within the first 30k [06:53:17] i'll set up live streaming tomorrow :) [06:53:26] it's already dawning :/ [07:01:59] ah, it's 07:00 here, but i already had some hours of sleep during the night :) [07:22:33] ETA of my hddscan2: friday, at about this time of the day [07:22:50] it's ~53% done, based on sector count [07:24:12] i've calculated the total hours needed as 151, based on the current speed of mine [08:03:54] *** Quits: [Saint] (~saint@rockbox/user/saint) (Quit: No Ping reply in 180 seconds.) [08:03:54] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [08:03:54] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [08:05:48] *** Joins: [Saint] (~saint@rockbox/user/saint) [08:09:48] * user890104 just got an awesome idea - parsing hddscan2's "state" array instead of generating a png image out of the framebuffer [14:03:54] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [14:03:54] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [16:02:08] * TheSeven will probably just write a general purpose ipod screen streaming tool :) [16:02:24] and downloading the state array will take quite a while [16:09:31] ah, you are allocating one byte for each sector [16:10:03] using ~28 MB for the array? [16:10:17] yes [16:10:27] the ram is pretty much maxed out for 160GB drives [16:10:37] why don't you pack 4 sector states in a single byte? [16:10:53] you're such a RAM waster :) [16:11:04] because I need at least 4 bits per sector [16:11:31] * user890104 checks the source again [16:11:32] and the other 4 are filled with a "last touched" pass number to help analyzing the result [16:11:46] aren't there about 10 possible states in total? [16:12:00] 14 [16:12:18] ah, now (state & 0x0F) makes sense :) [16:13:29] isn't looping over the array a bit slow, every time you show the progess? [16:13:45] i mean generating count[] from state[] [16:14:45] do I care? ;) [16:14:51] haha [16:15:06] it's done only every few seconds and the CPU is otherwise mostly idle [16:15:21] <[Saint]> Its not something that needs every bit of efficiency squeezed from it. [16:15:47] it also has the advantage that you can backup and restore its state by just downloading/uploading that memory area [16:15:53] without the code needing to be aware of it [16:15:57] * [Saint] plays a game of "Don't kill me for mentioning real Apple stuff": [16:15:58] well, if it doesn't affect the overall speed of the scanner, it's ok for me [16:16:14] <[Saint]> Does anyone know of a way I can restore an iPod Touch without iTunes, on Linux? [16:16:31] depends on which level you're at. [16:16:44] <[Saint]> Which level? [16:16:49] to get it from DFU through WTF to iRecovery you could use our ipoddfu.py script [16:16:55] but no idea how to talk to iRecovery [16:17:33] * [Saint] discovers iDevicerestorer [16:17:40] http://theiphonewiki.com/wiki/IRecovery [16:17:43] <[Saint]> ...that doesn't fucking support IPT [16:17:48] <[Saint]> Bleargh! [16:18:45] how is iPT different from the iPhone for these matters? [16:19:05] <[Saint]> I suspect they simply haven't added the V/PID? [16:19:38] <[Saint]> Ooooh, found one with exactly the same name that does. [16:20:08] <[Saint]> Hahaha...there's not one, not two, but three different "idevicerestore(er)" projects. [16:20:12] <[Saint]> Popular name. [18:39:44] *** Joins: n1s (~n1s@rockbox/developer/n1s) [19:30:23] user890104: http://theseven.bounceme.net/~theseven/tmp/ipodstream.htm ;) [19:34:59] * [Saint] loves the way TheSeven does this [19:35:10] * TheSeven fixes up the server-side code to use less CPU [19:35:11] <[Saint]> "Maybe might be useful..." [19:35:16] <[Saint]> [19:35:21] <[Saint]> "Here's " [19:49:19] who's curious how it works? :) [19:51:49] *** Quits: n1s (~n1s@rockbox/developer/n1s) (Ping timeout: 248 seconds) [20:03:54] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [20:03:54] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [20:13:48] TheSeven: me [20:14:07] wtf? websocket + canvas? [20:14:24] uhm ... you "ported" my php script to JS :) [20:15:14] and you're gzipping it... [20:19:47] and why are you XORing the current buffer with the incoming data? [20:22:08] ah ... it seems to compress pretty well :) [20:36:48] user890104: the server side is written in python, directly interfacing with libemcore [20:40:16] i see [20:40:27] [20:19:43] and why are you XORing the current buffer with the incoming data? [20:42:27] so that I only have to transfer the difference, which compresses much better, especially if the screen content's are mostly static [20:42:49] if the screen contents stayed the same, i'm gzipping an array of zeroes, which will only be a handful of bytes to transmit [20:43:06] ah, nice [20:43:19] what I'm doing is basically a simplified version of what PNG is doing, along with differential compression [20:43:27] how frequently are you dumping the framebuffer? [20:43:30] PNG also compresses gradients which I didn't care about here [20:43:34] every second [20:43:50] mine does downloadfile every 5 seconds [20:46:07] but it connects/disconnects every time [20:46:43] since the loop is implemented using a batch script (because i can't write it in python) [20:49:02] def streamer(): [20:49:02] global screenbuf, diff, wakeup [20:49:02] while True: [20:49:02] oldbuf = screenbuf [20:49:03] screenbuf = bytearray(emcore.read(baseaddr, size)) [20:49:05] diff = numpy.bitwise_xor(oldbuf, screenbuf) [20:49:07] with wakeup: wakeup.notify_all() [20:49:09] time.sleep(1) [21:01:26] @echo off [21:01:26] :start [21:01:26] "d:\Programs\Portable Python\App\python.exe" d:\Work\freemyipod\svn\emcore\trunk\tools\emcore.py downloadfile 0x09C0D068 0x00038400 d:\Misc\Web\framebuffer\fb4 [21:01:26] timeout /t 5 /nobreak [21:01:26] goto start [21:01:38] :) [21:04:14] hm... if I could grab the actual screen image somehow... [21:04:57] would get me rid of that annoying flicker [21:09:17] in terms of reading from the lcd controller? [21:10:23] hm, is there another option? [21:21:26] <[Saint]> Of course, you know what the obvious next step is, dontcha? [21:21:40] <[Saint]> emCORE Remote Terminal [21:21:48] <[Saint]> :) [21:25:10] TheSeven: does hddscan2 need a usb power source, in order to work? [21:36:22] user890104: well, it won't survive days on battery I guess? :) [21:36:43] i mean, does hddscan pause when the power is unplugged? [21:36:56] it does pause when the battery goes below a certain level [21:36:59] there was a "pause" variable somewhere in the source [21:37:06] ah, exactly [21:37:57] because i need my iCable in order to connect n2g and try to make a umsd out of umstest :) [21:38:11] (how about that name?) [21:41:48] the MSC protocol only needs raw access to the storage media in order to work, right? [21:44:05] yes [21:44:18] so it needs to unmount it first and remount it afterwards [21:44:26] otherwise you'll get funny corruption [21:46:53] ah, i see [21:46:57] *** Joins: n1s (~n1s@rockbox/developer/n1s) [21:47:08] i'm glad i learned that the easy way :) [21:49:08] let's see if virtualbox's usb driver is going to make my win7 bluescreen again [21:51:49] *** Quits: n1s (~n1s@rockbox/developer/n1s) (Ping timeout: 248 seconds) [22:08:09] *** Quits: [Saint] (~saint@rockbox/user/saint) (Remote host closed the connection) [22:09:08] *** Joins: [Saint] (~saint@rockbox/user/saint) [22:13:40] * user890104 fails to build umsboot ... again [22:14:07] why do you even need to? [22:14:45] i'm trying to build a complete installer, in order to update emcore on my n2g [22:15:14] http://pastie.org/pastes/8254310/text [22:15:58] CROSS=arm-none-eabi- and the TNT toolchain's bins dir is first in PATH [22:19:15] try CFLAGS=/usr/include/i386-linux-gnu [22:19:42] stupid system includes... [22:19:49] er, CFLAGS=-I/usr/include/i386-linux-gnu [22:22:06] thanks, it worked [22:22:25] <[Saint]> i386? [22:23:36] <[Saint]> Silly me thought 32bit stuffs was basically dead [22:25:48] my VM is 32-bit, i can't run a 64-bit OS in a VM, on a CPU that doesn't support hardware virtualisation (like mine) [22:26:35] <[Saint]> ancient CPU? [22:26:56] <[Saint]> or just unlucky? [22:27:11] well, it's almost 4-5 years old [22:27:33] <[Saint]> I have several that are older, that do. [22:28:34] <[Saint]> Hmmm, what's in this thing... [22:29:11] intel pentium t4200 @ 2.00 GHz [22:29:42] <[Saint]> AH, that's right. A bunch of old AMD 64 X2 4200+ [22:31:12] <[Saint]> 2008ish, iirc [22:31:23] <[Saint]> No, 06? [22:31:30] <[Saint]> somewhere around there. [22:31:45] <[Saint]> 4 of the fuckers sure can put out some heat, I know that much. [22:34:23] TheSeven: any idea why i'm unable to format umstest as NTFS? [22:34:32] "Windows was unable to complete the format" [22:35:03] How big is that volume? [22:35:20] 1 MB :) [22:35:25] Right [22:35:38] I would very much *not* be surprised if that's too small for NTFS :) [22:39:45] https://mail.gnome.org/archives/commits-list/2013-May/msg08176.html [22:41:41] Although it's hard to find specific data [22:44:53] * user890104 adds an extra sector to the RAMDISK :) [22:47:03] No promises! [22:47:17] TheSeven: is there a graceful way to end umstest? [22:47:31] Other sources suggest 10MB, or 20MB, or 30GB (those confuse NTFS minimal sizes with Windows minimal sizes)... [22:47:59] killthread'ing it and then doing emcore.py ls, seems to rewrite something in the RAM, and leads to Undefined instruction [22:52:17] *** Quits: [Saint] (~saint@rockbox/user/saint) (Remote host closed the connection) [22:53:12] *** Joins: [Saint] (~saint@rockbox/user/saint) [22:56:51] user890104: if you haven't re-mounted storage, ls should fail [22:57:10] a graceful way to exit the test app is either ejecting the mass storage drive, or unplugging the ipod [22:58:14] hm... not sure what to think about this drive... http://theseven.bounceme.net/~theseven/tmp/ipodstream.htm [23:10:17] *** Quits: [Saint] (~saint@rockbox/user/saint) (Remote host closed the connection) [23:11:15] *** Joins: [Saint] (~saint@rockbox/user/saint) [23:18:07] hard-resetting the ipod, after it crashed, causes ... guess what ... [23:18:11] hint: it's blue :) [23:19:12] even checkdisk wanted to have a look at C: [23:20:00] TheSeven: can you share your screen streamer's source? [23:20:22] http://paste.pm/9a8.js [23:20:33] + libemcore + the html/js files that you can access anyway [23:23:31] ah, you implemented a websocket server yourself :) [23:23:47] i thought you used some library for that