ioFTPD v6.6.1 BETA Released - Page 2

Flow · 06-05-2008, 03:54 PM

ok, solved, found the issue. Im not gonna admit that i overwrote the /etc folder. Must be the damn cat, so problem solved. The issue was a nooby user. Sorry ....

thomas74 · 06-11-2008, 10:59 PM

Is the lockup bug fixed in either 5.0 or 6.0?

Yil · 06-12-2008, 05:13 AM

Unsure. I removed a socket call that I thought might be causing problems so it's possible it's fixed. It's a very rare bug though and nearly impossible to reproduce so only time will tell.

thomas74 · 06-13-2008, 12:26 PM

it really annoys me to inform the lockup bug is still there. I use 6.0 with latest nxtools and ioninja on windows vista.

What's the latest version of ioFTPd that does not contain this bug?

Flow · 06-13-2008, 09:54 PM

Lockup ? How do i reproduce this lockup? I wanna have a lockup too

thomas74 · 06-14-2008, 01:06 AM

I wish I could tell what's causing the lockup. The machine I'm running the server on is a fast 100 mbit on OS vista. If I could be of any help I'll volunteer, the lockup on this machine happens often.

And again, what release of io does not contain this bug

Flow · 06-14-2008, 02:06 AM

Maybe Yil can make you a -debug version to resolve this issue. I myself have been using/participate with ioFTPd almost every new version without this lockup issue. You mention lockup often... your hardware are ok? HDD and MEM specially.

thomas74 · 06-14-2008, 02:41 AM

I'm fairly certain my hardware is OK. I have not found a pattern as to why this bug occur, but fast transfers and several bilateral threads seem to be ingredients needed to make it occur. Although, the bug might occur on little load with just one thread going. It also seems the bug will occur when there are directory changes. I don't think I've seen it occur during a transfer in progress, but only when the transfer goes from CD1 to CD2 or to Sample. However, the directory change will cause a lockup on other simultaneous going threads.

This is indeed frustrating, and I hope it gets sorted out. I'm just too effin' lazy to start learning a new ftp-soft.

.

Yil · 06-14-2008, 09:47 AM

thomas: Describe exactly what you see happening. Can you issue "site crashnow" and does it work? If you try to connect to the server immediately after you see it acting wierd do you at least get as far as the login prompt, an accepted connection, or does the connection just time out? And describe your hardware. I don't think you have a hardware problem, but I was able to see this bug on multi-processors in the past more often so that's useful to know...

Microsoft offers a tool called app-verifier. It's freeware. Try downloading that and setting it up to monitor ioFTPD. It will make ioFTPD use a lot of memory and be a bit slower but it can catch a lot of issues. If it finds any let me know. There are 1-2 false positives, but for the most part if it complains it's something that needs fixing. You might need to download MS Tools for Debugging in order to get windbg to create the stack traces app-verifier requests to assist in debugging, so grab that as well... This is the single best way to help solve this issue.

Ordinarily I'd say just send me a crashnow dump so I can see what's going on, but because the process is so badly messed up that won't work. You might be able to use the vista debug/memdump or whatever option from task manager to create a minidump of the process. I can use that to verify it's the same problem I've been trying to track down. This is actually far less useful than the app-verifier tracking because that might catch the problem when it occurs and not when it shows up later.

Yil · 06-14-2008, 09:54 AM

Oh, as far as last version without this issue. I have no idea and believe me I wish I did... I'm pretty convinced it's been there forever. It's the fact that we started getting multi-processor and faster machines that is making it show up now...

Assuming you aren't running recent versions of ioNiNJA/ioSFV you could always try going all the way back to 585r just to prove it didn't occur there if it really does happen every week or so to you...

thomas74 · 06-14-2008, 10:49 AM

Quote:

Originally Posted by Yil

thomas: Describe exactly what you see happening. Can you issue "site crashnow" and does it work? If you try to connect to the server immediately after you see it acting wierd do you at least get as far as the login prompt, an accepted connection, or does the connection just time out?

1) I send a release to the server containing two cds(two folders) and a sample(one folder). The dirs are always created successfully, and if I manage to get the SFV in, it seems like all files in the directory will transfer. It would seem that the server locks up during the directory changes. Let's say CD1 was transferred successfully and I start uploading the sample. I make the directory SAMPLE, but rather than completing the transfer I get a message similar to "no connection could be made"; and the same message is repeated until the entire queue transferred unsuccesfully. When the bug already have occured and I try to upload something, the transfer will just stall for minutes, hours or days, until I stop it -- no errors. I am of the impression that this bug occurs only during directory changes.

2) I can issue SITE CRASHNOW, but I don't see any dump being created. I see a crashlog being created, but I'm not sure if it's due to my ctrl+alt+del ending of ioFTPD or me issuing SITE CRASHNOW. From the crashlog:

Quote:

Fri Jun 13 23:08:08 2008 - ioFTPD v6.6.0
Unhandled exception: Access Violation (0xC0000005)
Address: 0x0042BFDD [attempting to write data to 0x00000000]
PID=7044, PATH=C:\ioFTPD\system\ioFTPD.exe
Thread ID: 4836

System information:
Processor #0 Name: Intel(R) Core(TM)2 Duo CPU T5450 @ 1.66GHz
Processor #0 Identifier: x86 Family 6 Model 15 Stepping 13
Processor #1 Name: Intel(R) Core(TM)2 Duo CPU T5450 @ 1.66GHz
Processor #1 Identifier: x86 Family 6 Model 15 Stepping 13
OS: Windows 6.0 (build 6000)
Registry: Windows Vista (TM) Home Premium
Decoded: Vista Home Media Center Edition
Page size: 4096
Notice: Unable to acquire Debug Rights

3) Usually I am able to login after the lockup has occured, I can also issue various commands. Other times again, I'm not able to log in at all; the server won't connect to the client at all. I haven't been able to see any pattern.

Quote:

Originally Posted by Yil

And describe your hardware. I don't think you have a hardware problem, but I was able to see this bug on multi-processors in the past more often so that's useful to know...

Yes, as you can see above it's a core2duo with 2 gigabyte or RAM.

Quote:

Originally Posted by Yil

Microsoft offers a tool called app-verifier. It's freeware. Try downloading that and setting it up to monitor ioFTPD. It will make ioFTPD use a lot of memory and be a bit slower but it can catch a lot of issues. If it finds any let me know. There are 1-2 false positives, but for the most part if it complains it's something that needs fixing. You might need to download MS Tools for Debugging in order to get windbg to create the stack traces app-verifier requests to assist in debugging, so grab that as well... This is the single best way to help solve this issue.

Ordinarily I'd say just send me a crashnow dump so I can see what's going on, but because the process is so badly messed up that won't work. You might be able to use the vista debug/memdump or whatever option from task manager to create a minidump of the process. I can use that to verify it's the same problem I've been trying to track down. This is actually far less useful than the app-verifier tracking because that might catch the problem when it occurs and not when it shows up later.

Alright, I will try installing app-verifier to see if anything can be found.

Quote:

Oh, as far as last version without this issue. I have no idea and believe me I wish I did... I'm pretty convinced it's been there forever. It's the fact that we started getting multi-processor and faster machines that is making it show up now...

Assuming you aren't running recent versions of ioNiNJA/ioSFV you could always try going all the way back to 585r just to prove it didn't occur there if it really does happen every week or so to you...

Hehe, could be worth the try. This bug occurs several times a week, and if I wanted I'm sure I could make it occur several times a day. Trouble however, is that I don't have physical nor complete remote access to the vista machine.

Yil · 06-14-2008, 02:13 PM

Cool. Let me know what app-verifier turns up

Just to clarify something. If you enter "site crashnow" does the process actually exit?

After the lockup occurs a critical low level windows lock is somehow held. This prevents new processes, threads, and sockets from being created. However 10 sockets are pre-allocated for accepting new client connections so for a little bit you can actually connect and login (provided it's an IP match for the hostname because the server can't reverse resolve at this point) but you can't transfer any directory listings or files over a data channel. You can probably use stat -l and still get directory listings over the control connection though. This should also be true of your current connection after it locks up...

exitwin98 · 06-15-2008, 12:29 AM

use "site swho username" ioFTPD will crash

CRASH-Log.txt

Code:

Sun Jun 15 16:52:23 2008 - ioFTPD v6.6.0
Unhandled exception: Access Violation (0xC0000005)
Address: 0x0041900F [attempting to read data from 0x00000000]
PID=5328, PATH=D:\ioFTPD\system\ioFTPD.exe
Thread ID: 6000

System information:
Processor #0 Name: Intel(R) Pentium(R) Dual  CPU  E2140  @ 1.60GHz
Processor #0 Identifier: x86 Family 6 Model 15 Stepping 13
Processor #1 Name: Intel(R) Pentium(R) Dual  CPU  E2140  @ 1.60GHz
Processor #1 Identifier: x86 Family 6 Model 15 Stepping 13
OS: Windows 5.2 (build 3790)
Registry: Microsoft Windows Server 2003 R2
Decoded: Server 2003 R2 Enterprise Edition - Service Pack 2
Page size: 4096

Modules:
--------
[00400000 - 0049d000]: D:\ioFTPD\system\ioFTPD.exe (v6.6.0.0)
[7c930000 - 7ca00000]: C:\WINDOWS\system32\ntdll.dll (v5.2.3790.3959)
[7c800000 - 7c92b000]: C:\WINDOWS\system32\kernel32.dll (v5.2.3790.4062)
[10000000 - 10163000]: D:\ioFTPD\system\php4ts.dll (v4.4.7.7)
[77e10000 - 77ea0000]: C:\WINDOWS\system32\user32.dll (v5.2.3790.4033)
[77bd0000 - 77c18000]: C:\WINDOWS\system32\gdi32.dll (v5.2.3790.4237)
[77f30000 - 77fdb000]: C:\WINDOWS\system32\advapi32.dll (v5.2.3790.3959)
[77c20000 - 77cbf000]: C:\WINDOWS\system32\rpcrt4.dll (v5.2.3790.4115)
[76eb0000 - 76ec3000]: C:\WINDOWS\system32\secur32.dll (v5.2.3790.3959)
[71b10000 - 71b1b000]: C:\WINDOWS\system32\wsock32.dll (v5.2.3790.0)
[71b60000 - 71b77000]: C:\WINDOWS\system32\ws2_32.dll (v5.2.3790.3959)
[77b70000 - 77bca000]: C:\WINDOWS\system32\msvcrt.dll (v7.0.3790.3959)
[71b50000 - 71b58000]: C:\WINDOWS\system32\ws2help.dll (v5.2.3790.3959)
[774b0000 - 775e9000]: C:\WINDOWS\system32\ole32.dll (v5.2.3790.3959)
[775f0000 - 7767b000]: C:\WINDOWS\system32\oleaut32.dll (v5.2.3790.4202)
[4b750000 - 4b78d000]: C:\WINDOWS\system32\odbc32.dll (v3.526.3959.0)
[77370000 - 77407000]: C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_5.82.3790.3959_x-ww_78FCF8D0\comctl32.dll (v5.82.3790.3959)
[7ca10000 - 7d1ec000]: C:\WINDOWS\system32\shell32.dll (v6.0.3790.4184)
[77eb0000 - 77f02000]: C:\WINDOWS\system32\shlwapi.dll (v6.0.3790.3959)
[761a0000 - 761e7000]: C:\WINDOWS\system32\comdlg32.dll (v6.0.3790.3959)
[004a0000 - 005a4000]: D:\ioFTPD\system\tcl85t.dll (v8.5.2.2)
[760a0000 - 76131000]: C:\WINDOWS\system32\crypt32.dll (v5.131.3790.3959)
[76080000 - 76092000]: C:\WINDOWS\system32\msasn1.dll (v5.2.3790.3959)
[77b60000 - 77b68000]: C:\WINDOWS\system32\version.dll (v5.2.3790.3959)
[76180000 - 7619d000]: C:\WINDOWS\system32\imm32.dll (v5.2.3790.3959)
[7f000000 - 7f009000]: C:\WINDOWS\system32\lpk.dll (v5.2.3790.3959)
[74ae0000 - 74b45000]: C:\WINDOWS\system32\usp10.dll (v1.422.3790.3959)
[77cd0000 - 77dd3000]: C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.3790.3959_x-ww_D8713E55\comctl32.dll (v6.0.3790.3959)
[00ba0000 - 00bb2000]: C:\WINDOWS\system32\odbcint.dll (v3.526.3959.0)
[03000000 - 03115000]: D:\ioFTPD\system\dbghelp.dll (v6.8.4.0)
[75870000 - 7592d000]: C:\WINDOWS\system32\userenv.dll (v5.2.3790.3959)
[71ba0000 - 71bf7000]: C:\WINDOWS\system32\netapi32.dll (v5.2.3790.3959)
[71a80000 - 71ac0000]: C:\WINDOWS\system32\mswsock.dll (v5.2.3790.3959)
[69660000 - 696b7000]: C:\WINDOWS\system32\hnetcfg.dll (v5.2.3790.3959)
[71a40000 - 71a48000]: C:\WINDOWS\system32\wshtcpip.dll (v5.2.3790.3959)
[76e30000 - 76e5b000]: C:\WINDOWS\system32\dnsapi.dll (v5.2.3790.4238)
[76ed0000 - 76ed7000]: C:\WINDOWS\system32\winrnr.dll (v5.2.3790.3959)
[76e70000 - 76e9e000]: C:\WINDOWS\system32\wldap32.dll (v5.2.3790.3959)
[76ee0000 - 76ee5000]: C:\WINDOWS\system32\rasadhlp.dll (v5.2.3790.3959)
[74800000 - 74807000]: C:\WINDOWS\system32\powrprof.dll (v6.0.3790.3959)


Threads:
--------
ID: 5600  [00130000-0012fbc4]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 77F45EDB -> [ADVAPI32 + 14EDB] ? LookupPrivilegeValueW() + 0x114
  # 3: 77F45F82 -> [ADVAPI32 + 14F82] ? LookupPrivilegeValueW() + 0x1BB
  # 4: 77F975AF -> [ADVAPI32 + 665AF] ? StartServiceCtrlDispatcherA() + 0x93
  # 5: 004487C8 -> [ioFTPD + 477C8] WinMain() + 0x48
                    [c:\projects\ioftpd6\6.5.1\src\main.c, line 483]
  # 6: 00406AEE -> [ioFTPD + 5AEE] __tmainCRTStartup() + 0x113
                    [f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c, line 263]
  # 7: 7C82F23B -> [kernel32 + 2E23B] ? ProcessIdToSessionId() + 0x209


ID: 3168  [00cd0000-00ccff28]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 0044CF11 -> [ioFTPD + 4BF11] ProcessMessages() + 0x41
                    [c:\projects\ioftpd6\6.5.1\src\internalmessagehandler.c, line 106]
  # 3: 00448675 -> [ioFTPD + 47675] CommonMain() + 0x35
                    [c:\projects\ioftpd6\6.5.1\src\main.c, line 340]
  # 4: 00448740 -> [ioFTPD + 47740] ServiceMain() + 0xA0
                    [c:\projects\ioftpd6\6.5.1\src\main.c, line 457]
  # 5: 77F45E91 -> [ADVAPI32 + 14E91] ? LookupPrivilegeValueW() + 0xCA
  # 6: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 4412  [00fd0000-00fcfed4]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041BCE0 -> [ioFTPD + 1ACE0] TimerThread() + 0x2B0
                    [c:\projects\ioftpd6\6.5.1\src\timer.c, line 141]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 1344  [01330000-0132ff0c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C45A -> [ioFTPD + 1B45A] WorkerThread() + 0x5A
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 441]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 5232  [01430000-0142ff0c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C45A -> [ioFTPD + 1B45A] WorkerThread() + 0x5A
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 441]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 4444  [01530000-0152ff0c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C45A -> [ioFTPD + 1B45A] WorkerThread() + 0x5A
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 441]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 3948  [01730000-0172ff4c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 0041C280 -> [ioFTPD + 1B280] IoThreadEx() + 0x20
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 561]
  # 3: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 4468  [01830000-0182ff4c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 0041C280 -> [ioFTPD + 1B280] IoThreadEx() + 0x20
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 561]
  # 3: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 5852  [01930000-0192ff14]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C1A4 -> [ioFTPD + 1B1A4] EncryptionThread() + 0x14
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 651]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 5012  [01a30000-01a2ff14]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C1A4 -> [ioFTPD + 1B1A4] EncryptionThread() + 0x14
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 651]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 660   [01b30000-01b2fed8]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C8024ED -> [kernel32 + 14ED] ? Sleep() + 0xF
  # 3: 00424C15 -> [ioFTPD + 23C15] SocketSchedulerThread() + 0x4A5
                    [c:\projects\ioftpd6\6.5.1\src\socket.c, line 1774]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 3180  [01c30000-01c2fcec]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 5880  [020f0000-020eff7c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 5984  [021f0000-021eff0c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C6E8 -> [ioFTPD + 1B6E8] WorkerThread() + 0x2E8
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 519]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 1408  [00a30000-00a2ff0c]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 7C821C8D -> [kernel32 + 20C8D] ? WaitForSingleObject() + 0x12
  # 3: 0041C45A -> [ioFTPD + 1B45A] WorkerThread() + 0x5A
                    [c:\projects\ioftpd6\6.5.1\src\threads.c, line 441]
  # 4: 7C824829 -> [kernel32 + 23829] ? GetModuleHandleA() + 0xDF


ID: 6000  [011d0000-011cd9c8]
  # 1: 7C9585EC -> [ntdll + 275EC] ? KiFastSystemCallRet() + 0x0
  # 2: 0153DC10 -> [[unknown module]]
  # 3: 7C956D2B -> [ntdll + 25D2B] ? NtClose() + 0xC
  # 4: 03062123 -> [DbgHelp + 61123] ? MiniDumpReadDumpStream() + 0x6703
  # 5: 0305D50D -> [DbgHelp + 5C50D] ? MiniDumpReadDumpStream() + 0x1AED
  # 6: 03063089 -> [DbgHelp + 62089] ? MiniDumpReadDumpStream() + 0x7669
  # 7: 030657A9 -> [DbgHelp + 647A9] ? MiniDumpReadDumpStream() + 0x9D89
  # 8: 7C95779B -> [ntdll + 2679B] ? ZwReadVirtualMemory() + 0xC
  # 9: 7C80232D -> [kernel32 + 132D] ? ReadProcessMemory() + 0x1B
  #10: 03062871 -> [DbgHelp + 61871] ? MiniDumpReadDumpStream() + 0x6E51
  #11: 03062920 -> [DbgHelp + 61920] ? MiniDumpReadDumpStream() + 0x6F00
  #12: 0303B953 -> [DbgHelp + 3A953] ? SymEnumSourceFilesW() + 0xF3



-------------------------------------------------------------------------------

Yil · 06-15-2008, 11:27 AM

exitwin98: I can't reproduce that problem. Does it happen every time for you? Does it only happen for some users? Did the logfile show a user being deleted from the server around the same time? I'll PM you a server to upload the minidump to. The stack trace was unhelpful.

Yil · 06-15-2008, 04:23 PM

exitwin98: I found the problem from the minidump you sent me. God, I love having added that feature. I'll get a new version out pronto to fix the issue along with a couple other things isteana pointed out to me.