[00:07:29] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [00:08:22] *** Joins: bcobco (~bcobco@77.225.204.119) [01:06:45] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [01:07:12] *** Joins: bcobco (~bcobco@77.225.204.119) [01:32:44] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [01:33:07] *** Joins: bcobco (~bcobco@77.225.204.119) [02:10:18] user890104: hm... I'm running into USB trouble :/ [02:10:32] with double buffered writes I get USB lockups [02:10:54] VERY similar to what we've been observing with the old driver [02:11:09] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [02:11:09] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [02:11:34] if I schedule a new RX txn very shortly after the old one completes, the new one never finishes [02:43:01] user890104: seems like starting a new transfer on an endpoint before acknowledgeing the old IRQ is problematic on this core [03:19:18] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [03:19:41] *** Joins: bcobco (~bcobco@77.225.204.119) [05:59:36] *** Quits: [Saint] (~saint@rockbox/staff/saint) (Remote host closed the connection) [06:00:40] *** Joins: [Saint] (~saint@rockbox/staff/saint) [06:35:07] *** Quits: TheSeven (~quassel@rockbox/developer/TheSeven) (Ping timeout: 252 seconds) [06:36:25] *** Joins: TheSeven (~quassel@rockbox/developer/TheSeven) [08:07:46] *** Quits: [Saint] (~saint@rockbox/staff/saint) (Read error: Connection reset by peer) [08:08:45] *** Joins: [Saint] (~saint@rockbox/staff/saint) [08:11:09] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [08:11:09] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [10:06:13] TheSeven: OSX also seems to be smart enough to issue read requests for many sectors at once [10:06:36] i'll just bump the buffer to a couple of megabytes, just to make it work [10:07:33] i have a bunch of computers at the office running various operationg systems, so i'd like to test it on as many devices as possible [10:14:19] btw: UMSboot on OSX: http://paste.pm/g99.js [10:14:26] and it mounts fine [10:56:54] disk mode: http://paste.pm/g9a.js [10:57:34] (i feel guilty for allocating 56 MB of RAM for storage buffer, just to be on the safe side :P) [11:02:56] TheSeven: we need to skip .* (dot-files) when looking for .ubi in UMSboot. OSX stores some extended FS attributes in ._ files [12:12:09] user890104: huh? [12:13:30] do they indeed store those with "UBI" extension in the 8.3 name? [12:19:43] i guess so [12:20:24] i also have .Trash, ._Trash, .spotlight-something and .fsevent-something [12:20:27] osx tries to store that things by default everywhere. but if i remember well that can be disabled using diskutil [12:20:59] the spotlight folder can be avoided if there's an empty file named .metadata_never_index in the root dir [12:29:57] http://blog.hostilefork.com/trashes-fseventsd-and-spotlight-v100/ [12:31:59] well I don't really care about folders - we've taken care of those now [12:46:53] i suggest skipping all files whose name begins with a dot [13:02:34] how can you do that? [13:02:51] those dots don't appear in the 8.3 name [13:03:04] and I don't really want to implement a full LFN stack just because OSX is dumb [13:08:37] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [13:10:42] *** Joins: bcobco (~bcobco@77.225.204.119) [13:33:16] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [13:33:17] TheSeven: can you examine a disk image if i send you one? [13:33:41] i can upload an installer in umsboot, then dd the whole image to a file [13:33:41] *** Joins: bcobco (~bcobco@77.225.204.119) [13:37:49] yes, I'll have a look at that later... [13:38:06] currently debugging a rather nasty USB lockup in diskmode [13:38:37] it's generally only writes that lock up, and I'm not quite sure yet what's happening [13:42:58] i have been copying about 30 GB of mixed content (mostly music) using the current version of disk mode (with my buffer patch) [13:43:16] it started at about 10 MB/s, and now it's down to about 3 MB/s [13:43:25] probably because of bad sectors [13:43:35] it isn't using the bad block table, right? [13:47:23] it should be using it if one exists [13:48:02] that would also explain a performance drop (because of seeks and "slow" sectors around the damaged ones) [13:48:22] if you'd run into bad ones, it would typically drop to a few KB/s [13:48:28] that's handled by our ATA driver, right? [13:48:35] yes [13:48:43] so the storage API is at a higher level [13:48:47] ah, i see [13:49:13] so you're not having any USB transfer problems? weird... [13:49:27] but since i have already run hddscan2 on the whole drive, i guess i have a bbt and it's using it [13:49:37] for me every transfer of ~10MB locks up at some point [13:49:45] uhm no [13:49:53] that's on w7x64 [13:50:01] with the buffer patch [13:50:10] hm, I'll need to try an unmodified diskmode app... [13:50:43] that's the only thing i've changes: [13:50:46] static uint8_t __attribute__((aligned(32))) storage_buffer[0x3800000]; // 56 MB [13:52:13] I've fixed one (kernel) bug already that should have hit yours as well... [13:52:56] well my ipod's kernel hasn't crashed yet [13:52:56] there can be lost USB IRQs if you schedule something on an endpoint before clearing the IRQ flag of it [13:53:02] and the transfer is complete [13:53:40] now i'll attach it to the mac so i can dump umsboot's data [13:55:16] would you like to setup your svn client so you can commit changes? [13:55:58] maybe tonight... first I'll have to actually fix these things ;) [13:56:21] * TheSeven hates the fact that we don't have an IRQ safe debugging console [14:01:01] can you use UART? :) [14:02:55] i have a modified dock connector cable that supports both uart and usb (and charging of course) [14:03:10] and a cp2102-based uart-to-usb adapter [14:03:52] the console api generally isn't irq safe [14:04:03] and will lead to lockups more often than not [14:11:08] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [14:11:08] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [14:11:33] hm, using lcdconsole_*_noblit from IRQ mode and periodically aclling lcdconsole_update from usermode should reduce the problem to possible corruption and avoid lockups [14:14:58] i was talking about accessing uart directly [14:16:19] probably much slower than LCD [14:34:18] the right file: INSTAL~1UBI [14:34:18] the wrong file: _INSTA~1UBI [14:34:28] * user890104 starts to hate osx [14:34:46] here's the whole image: http://www.datafilehost.com/d/383baf54 [14:38:15] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [14:38:40] *** Joins: bcobco (~bcobco@77.225.204.119) [14:42:09] directory listing: http://paste.pm/g9e.js [15:09:50] gevaerts: have you ever seen a host issue a WRITE(10) with zero sectors size? [15:10:51] hmmm [15:11:26] Not that I remember [15:11:34] It's an interesting case :) [15:11:56] correct reaction is to immediately reply with a success CSW I guess? [15:13:12] There's no special handling for it in the rockbox code, so there we first wait for an empty USB transfer, and then (if al goes well!) we do a zero-size write to disk [15:13:22] I can think of several place where that might go wrong... [15:13:43] An immediate CSW *might* be correct [15:13:50] * gevaerts doesn't remember how USB works :( [15:14:15] If there's an explicit empty transfer on the bus, not waiting for that is going to be an issue [15:14:26] But I don't remember if you can even have those in bulk [15:14:49] Are you seeing such beasts? [15:17:17] From the SCSI SBC2 spec, "A TRANSFER LENGTH field set to zero specifies that no logical blocks shall be written. This condition shall [15:17:20] not be considered an error" [15:17:33] So it's definitely something that is allowed to happen [15:18:51] Interestingly enough, for WRITE(6) (which is de-facto obsolete and which I don't even handle in rockbox), zero sectors meant 256 [15:31:01] I don't think you can even do a 0 bytes bus transfer [15:32:40] You can do arbitrary writes with sg_write_long. I'll have a go tonight to see what happens in rockbox [15:32:40] I ran into this situation, but it's looking more like the sector count was trashed on my side somewhere [15:33:08] It should be trivial to handle properly, as soon as we know what "properly" means exactly [15:33:48] according to wireshark that write was actually 1 sector, but somehow my code doesn't believe that. hm... [15:34:17] *** Quits: [Saint] (~saint@rockbox/staff/saint) (Read error: Connection reset by peer) [15:35:15] *** Joins: [Saint] (~saint@rockbox/staff/saint) [15:42:27] CBW: 55 53 42 43 C2 07 00 00 00 10 00 00 00 00 0A 2A 00 00 00 4A 72 00 00 01 00 00 00 00 00 00 00 [15:42:42] why does it think that this would be zero bytes [15:42:56] the device receives the CBW intact [16:44:03] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [16:44:29] *** Joins: bcobco (~bcobco@77.225.204.119) [17:13:15] hm. I'm dealing with cache pollution here... [17:13:32] something reads the CBW buffer into a cacheline when it shouldn't [17:18:31] user890104: getting about 30% better read performance and 70% better write performance with my code compared to yours [17:18:41] so double buffering is definitely worth it [17:27:25] user890104: https://mega.co.nz/#!xt8gAQgL!l_PTSD3GoFmLC6jk8f6pzsKXGBuFGEJgMuKlH2v4Hpo [17:27:33] https://mega.co.nz/#!os1HkaTA!KVEATHpZWrUosiX38SHZtKTVM2l2sEKnIX309vMy1kY [17:28:22] can you do a few performance and stress tests on those? [17:28:37] i.e. do reads/writes actually end up where they should etc. [17:46:11] * gevaerts doesn't have a classic, or he'd join the competition! [17:46:49] TheSeven: how are you measuring? dd times? [17:47:01] copying files in nautilus and watching dstat [17:47:17] ah, ok [17:47:24] Fairly real-world then :) [17:48:42] TheSeven: ah, you didn't like my explanatory messages :) [17:50:18] wow! reading large files gets to about 30 MB/s! [17:51:07] about 25 MB/s for smaller files [17:58:05] hm, more like read 13MB write 18MB on my HDD [17:58:20] but we all know that this particular disk is crap [18:05:00] * TheSeven just built rockbox and doesn't notice any USB problems with that build [18:08:37] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [18:09:01] *** Joins: bcobco (~bcobco@77.225.204.119) [18:10:02] <[Saint]> A user reported that USB works with git head on N2G IFF USB is plugged when the device is powered down. [18:10:24] <[Saint]> Oddly, I could replicate this. [18:10:57] <[Saint]> Success in approximately 4 of 5 attempts. [18:11:28] <[Saint]> (compared to 5 of 5 failures with Rockbox up and running at connection time) [18:21:59] write here is about 14-17 MB/s [18:23:00] TheSeven: do you have read buffering now? Does it make much of a difference? [18:31:07] TheSeven: i am getting a real 30 MB/s read speed for flacs (a whole album is 300-350 MB, time is 10s) [18:32:52] uploading the same files varies between 20 and 22 MB/s [18:32:55] gevaerts: yes, that build has read buffering [18:33:03] so good job! :) [18:33:13] Does it make much of a real difference? [18:33:52] well, that's what probably explains this speed difference? ;) [18:34:41] the code just unconditionally reads 64K ahead of the request in the background [18:34:56] * gevaerts nods [18:35:25] can you upload the source code somewhere? [18:35:36] and if that read was completed by the time that the next request hits, the buffer is possibly used to satisfy the request, and another readahead is started on the other buffer [18:36:48] actually this code looks buggy in the case that the readahead doesn't finish before the next request comes in [18:37:10] The thing is of course that in rockbox, we also have sd targets that need a lot of cpu for the storage io, in which case it's not necessarily a win [18:37:17] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [18:37:31] in that case it probably reads the data twice [18:37:46] hm, or maybe not... could have got that right by accident [18:38:10] *** Joins: bcobco (~bcobco@77.225.204.119) [18:38:44] https://mega.co.nz/#!k9N1XD6I!T4yEnoNFwdm1CQGhwRgiYa5F7opfhmqJdB0ZGCHVHjs [18:38:53] contains a lot of rockbox code ;) [19:03:16] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [19:04:12] *** Joins: bcobco (~bcobco@77.225.204.119) [19:05:17] *** Quits: [Saint] (~saint@rockbox/staff/saint) (Remote host closed the connection) [19:06:16] *** Joins: [Saint] (~saint@rockbox/staff/saint) [20:11:11] *** Joins: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) [20:11:11] *** Quits: clustur (~logger@c-68-53-250-91.hsd1.tn.comcast.net) (Remote host closed the connection) [22:56:34] *** Quits: bcobco (~bcobco@77.225.204.119) (Remote host closed the connection) [22:56:59] *** Joins: bcobco (~bcobco@77.225.204.119) [22:57:12] TheSeven: does your version of diskmode look stable to you? [23:02:09] yes, but it needs more testing [23:02:40] I think line 209 (ums.c) should be removed [23:09:42] and maybe something else that checks if the sector is in range on a currently running read ahead [23:30:16] where was the kernel bug that you've found? [23:42:54] it seems like the USB stack doesn't like if the IRQ is not cleared before a new transfer is started on the EP [23:44:38] i guess you included the fix in your diskmode build? [23:44:44] yes [23:44:48] ok [23:51:15] i should be able to test it on a bunch of laptops tomorrow [23:51:37] we even have a laptop with chromium os IIRC [23:51:49] and some windows tablets