Running timing problems - "nan" time for a respons

scottaugust · Post by **scottaugust** » Fri Aug 30, 2002 6:12 pm

I have recently seen and been involved with several fluke timings for problems. Specifically 100, 371 and 102. Instead of getting a response of 0.000 for time I get "nan" for a response, which translates to 0.000 time in the rankings. The problem seems to have surfaced when the judge was down to run through all submissions to recreate the data base (my guess as to what was going on). For those of us that use this site to improve our skills in programming and problem solving, via competition, how can we tell if our solution is faster than someone else's, or one of our own previously sumbitted solutions. I think that this is important to fix!

I would like to hear if others feel the same way, or if I am in the minority.

Scott E August

Post by **fpnc** » Sat Aug 31, 2002 11:32 am

Please believe me we are the most interested in fix everything in this system. We are already working in a lot of aspects of the judge.

Best regards,

Ivor · Post by **Ivor** » Sat Aug 31, 2002 11:52 am

I don't doubt you're interested in it and are doing your best. (I make a deep bow). But Scott's right, this one is urgent -- it needs fixing and fast.

My main interest in solving the problems here is (was?) to have a chance to optimize my programs and try to beat others. But this timing problem has turned everything upside down.

It doesn't matter much if you fix it now or later, but when you fix it could I ask a favour -- to rejudge every submission starting from last weeken when the system was down. Please!

I do want to see a fair ranking.

Ivor

scottaugust · Post by **scottaugust** » Tue Sep 03, 2002 4:23 pm

I agree with Ivor, I have no doubt that you are continuing to update and fix the system. Could you give us a little insight as to what is going on and what some of the ideas for fixing thing are - you have a large resource out here of people that can give ideas/resources/code to fix some of these thing.

Scott

Post by **fpnc** » Tue Sep 03, 2002 4:55 pm

It seems that something turned wrong when our system went to the heaven_of_judging_systems thanks to some "person" - if we can name it that way. Some submissions need to be rejudged and we'll do it asap, I promise. It seems that new submissions are working as expected right now, isn't it? I hope it will be fixed soon, but before we have to be sure that it won't happen again in the future.

Best regards,

Ivor · Post by **Ivor** » Tue Sep 03, 2002 6:41 pm

No, new submissions are not working as they should! Somehow rare (!) 'nan' still comes through. The last 'nan' I got a couple of minutes ago on 640. See yourself -- I have an appropriate comment there.

There's still something fishy with timing.

Ivor

scottaugust · Post by **scottaugust** » Tue Sep 03, 2002 6:50 pm

I recieved a 'nan' as well on 628. Could you decribe your method of timing code execution? Is it just the unix "time" program, or are you doing some kind of messaging of the timing information. nan usually equates to a divide by zero, or something along those lines.

Let us know what we can do to help - if just waiting for you to fix it is the best we can do, we will do that.

Scott

Post by **fpnc** » Tue Sep 03, 2002 8:03 pm

I'm not totally sure how we monitor that information - I'm studying that part of the judge right now

- but I've seen the function wait3 that seems to return the user time and system time -- and we use that information to know the CPU time.

We must rejudge all the problems that suffer of that NaN problem. But we have a "big" problem: we don't know what are they. We suppose they start in one of the submissions sent the day the system went down, and it should finish in one of the submissions sent the day the system recovered. We don't know the exact submission # of those, so rejudging is quite a painful process for this. If you can help - providing a hint of what submissions # they are - we can try to do it faster.

Best regards,

Ivor · Post by **Ivor** » Tue Sep 03, 2002 8:24 pm

If you mean the ID numbers then it'd be hard to name those. And there are more of them than you might think. I suppose you could rejudge any 0.000 time that is likely to be of 'nan' origin. That is problems were 0.000 was submitted since server was down. Of course I mean problems where 0.000 time was not present before server was down. I could name a couple of problems: 100, 102, 640, 628, 492, 371, 449, 450... there are more I think, but I don't recall them atm.

I think that 'nan' is not the only problem. In 100 I can dare to doubt in 0.020 time -- it must be a damn good algorithm to get that time and I fear pascal is incapable of that. (I apologise if things are otherwise, no offence meant here, just personal opinion).

I know you rejudged some subissions to 100 and I saw how 0.000 times and couple of 0.040 times disappeared. However same people got 0.000 again. I fear it was 'nan'. I think mine was, but I never received a reply.

Ivor

Post by **fpnc** » Tue Sep 03, 2002 8:44 pm

We even thought our system was hacked because 0.000 with our input/output in p100 was very, very difficult... but, when I saw the way one of that persons reached the printf("xx\nxx\nxx\n...") solution, I was very... arggg! He was very clever, and he got it using allowed methods (and a lot of submissions, BTW)

Anyway, what I need to know is the first submission # that got NaN and the last one with that NaN problem - I don't bother if it was in problem 100, 640 or 371... We should rejudge every submission from that first one until that last one, because in that while, our system was not very accurate.

I'm involved in adding Freepascal support right now, and I've almost finished. When I finish that and also another couple of things in my "pending list" I'll try to rejudge them. Until that, the only thing I can say is... "sorry but you must wait

"

scottaugust · Post by **scottaugust** » Tue Sep 03, 2002 9:22 pm

The last time I received nan was today, submission number 1063807. I believe that Ivor received a nan the same day, not sure what his submission number was.

I don't believe that rejudging is the issue right now, when it happens - fine. The main issue is that there is still a problem in the timing method.

Post by **fpnc** » Tue Sep 03, 2002 9:27 pm

scottaugust wrote:The last time I received nan was today

->

I'll report again this issue to the others in this moment.

Ivor · Post by **Ivor** » Wed Sep 04, 2002 2:48 pm

Yes, 'nan' still happens, I got it too yesterday. So, the problem is still in the air somewhere.

We even thought our system was hacked because 0.000 with our input/output in p100 was very, very difficult... but, when I saw the way one of that persons reached the printf("xx\nxx\nxx\n...") solution, I was very... arggg!

Well that is unfortunate.

I have gone to some limits of actually testing the input -- to check the boundaries and such things. But I have never tried to get a printf solution. It's very.... I don't have a word for it. Anyway, you might just increase the input with small numbers, so not to affect running time very much. Like fitting in some 1000 testcases with numbers upto 1000. The performance wouldn't suffer much. And it will not fit in the sourcecode if you ensure that the output size is more than 40k.

Ivor

scottaugust · Post by **scottaugust** » Wed Sep 04, 2002 6:07 pm

I believe that the nan problem started when the judge restarted. I never saw or heard about the problem before the judge went to the great judge heaven.

Ivor has a very good idea to make the output of P100 larger than 40k - that is a nice solution to "printf" type of solutions.

Keep up the good work "judge people"!

Scott

Ivor · Post by **Ivor** » Tue Sep 10, 2002 5:41 pm

That's a tough request

But then again it's a nasty bug. It pops up rarely and I understand that it's hard to track. I'll try to give as much information as I can.

in what languages it happens

Not sure about it but I think it can happen in all languages. It sure does happen when C is used. Judging from problem 100, I think pascal is involved too (though there was a printf solution as in one post said).

if there were more submissions in the moment yours were judged

There were other submission at the time, including mine. Because the timing is not precise I sometimes submit 2-4 mails at a time. I can then see the boundaries of the running time.

if your program is doing something suspicious

An interesting question.

I don't believe it occupied with anything other than solving current task. Though I use register variables and rarely a couple of lines of asm, I don't think it should affect anything other than speed.

Also I'd like to know what % of submissions are "affected" with this sympthom

As I said it is a very rare problem. To get it you need a program that runs at the average speed below 0.060. Also many submissions are "needed". In process of optimization I change some little things (like the order of commands and so on...) so I submit many times. I guess 'nan' pops up once in 50 submissions. Not sure about that.

Please tell me also the exact place in which you see NaN (if email, what is the exact reply) and what is shown in ranklists and status...

I get a reply for every submission I make and it's only e-mail where it writes that program ran in 'nan' time. Exactly:

Your C program has solved Ok the problem 495 (Fibonacci Freeze)
in nan seconds with low memory spent.
Congratulations!

That's the last one I got. The online status page shows time 0.000 and so does the ranklist. Submission ID is 1077463. Right in the middle of the page: http://acm.uva.es/cgi-bin/OnlineJudge?S ... 55:1.00:60:

Hope this is of some use,
Ivor

OJ Board

Running timing problems - "nan" time for a respons

Running timing problems - "nan" time for a respons

Everything about 'nan'