## 860 - Entropy Text Analyzer

All about problems in Volume 8. If there is a thread about your problem, please use it. If not, create one with its number in the subject.

Moderator: Board moderators

yan8381
New poster
Posts: 4
Joined: Wed Nov 26, 2003 4:05 pm

### 860 - Entropy Text Analyzer

WA
I have test my c++ code for sample input ,that is correct. But I still got WA for this problem(#860) .

I have no more ideas what is wrong. .Can you help me?
Are there any suggested test cases ?
And could tell me any hints about this problem?!?

thanks

yan8381
New poster
Posts: 4
Joined: Wed Nov 26, 2003 4:05 pm

why there is no one help me?!? ~~~55555~~~

by the way, one more question:
I don't know what is the exactly meaning about "word".
for example, if the input is "don't" , Is it means is equal to "dont"
that is "don't","dont" or "do:,n!t" must be considered to same?!?
Or ther is another meaning?!?

how about a number, that is if the marks change to any number, like"do8293nt".

thank you for help.

little joey
Guru
Posts: 1080
Joined: Thu Dec 19, 2002 7:37 pm
Some tips:

A word is a string of chars, none of which is one of the following 14: ',', '.', ':', ';', '!', '?', '"', '(', ')', ' ', '\t', '\n', (char)EOF and '\0'. All other 256-14 characters can (and probably will) appear in a word.
So "don't", "do_not", "do\anot", "d19175", "do;nt" are all single, different words. Don't rely on library functions isspace(), isalnum(), iscntrl(), etc. !

"master_b" is the same as "Master_B", "mAsTeR_B", etc. but "****End_of_Text****" is not the same as "****END_OF_TEXT****". (Note there are 4 stars trailing this word, not 3 as in the output description).

There is no white space at the end of the input, the last '*' of "****END_OF_INPUT****" is immediately followed by EOF. My first program went into an endless loop waiting for the final white space that never came. (While I was thinking my algorithm was slow )

Hope it helps.

minskcity
Experienced poster
Posts: 199
Joined: Tue May 14, 2002 10:23 am
Location: Vancouver
It helped !!!!
it's really useful to read the problem description carefully...

suneast
New poster
Posts: 49
Joined: Sun Jan 17, 2010 6:25 am
Location: FZU
Contact:

### Re: 860 - Entropy Text Analyzer

I don't know why I am always getting WA...

can anyone give some hints to solve it?

suneast
New poster
Posts: 49
Joined: Sun Jan 17, 2010 6:25 am
Location: FZU
Contact:

### Re: 860 - Entropy Text Analyzer

yeah, after a lot times struggle...
I finally got AC

just output

Code: Select all

``````printf("%d %.1lf %.0lf\n",0,0.0,0.0);
``````
when there aren't any words in the test...and got ac...

Scarecrow
Learning poster
Posts: 69
Joined: Wed Oct 19, 2011 9:06 pm

### Re: 860 - Entropy Text Analyzer

someone please help me find the bug, or, provide some I/O where my code fails

Code: Select all

``````import java.io.*;
import java.util.*;

class Main
{
public static void main(String[] args) throws Exception
{
int count;
double lg, E_T;
String line, words[];
TreeMap<String, Integer> map = new TreeMap<String, Integer>();
Set<Map.Entry<String, Integer>> set;
BufferedReader input = new BufferedReader(new InputStreamReader(System.in));
BufferedWriter output = new BufferedWriter(new OutputStreamWriter(System.out));

while(!(line = input.readLine()).equals("****END_OF_INPUT****"))
{
map.clear();
count = 0;

if(!line.equals("****END_OF_TEXT****"))
{
do
{
words = line.split("[,.:;!?\"\\(\\) \\t]+");
count += words.length;

for(String s : words)
{
s = s.toLowerCase();
map.put(s, map.containsKey(s) ? map.get(s)+1 : 1);
}

}while(!(line = input.readLine()).equals("****END_OF_TEXT****"));

lg = Math.log10(count);
E_T = 0;
set = map.entrySet();

for (Map.Entry<String, Integer> entry : set)
E_T += (entry.getValue() * (lg - Math.log10(entry.getValue())));
E_T /= count;

output.write(String.format("%d %.1f %d\n", count, E_T, (int)(E_T*100/lg)));
}
else
output.write("0 0.0 0\n");
}

output.flush();
}
}``````
Do or do not. There is no try.

brianfry713
Guru
Posts: 5947
Joined: Thu Sep 01, 2011 9:09 am
Location: San Jose, CA, USA

### Re: 860 - Entropy Text Analyzer

Input:

Code: Select all

``````word1 word2

word3
****END_OF_TEXT****
****END_OF_INPUT****``````
AC output:

Code: Select all

``3 0.5 100``
Check input and AC output for thousands of problems on uDebug!

Scarecrow
Learning poster
Posts: 69
Joined: Wed Oct 19, 2011 9:06 pm

### Re: 860 - Entropy Text Analyzer

thanks brianfry713. but still getting WA. my updated code is -

Code: Select all

``````import java.io.*;
import java.util.*;

class Main
{
public static void main(String[] args) throws Exception
{
int count;
double lg, E_T;
String line, words[];
TreeMap<String, Integer> map = new TreeMap<String, Integer>();
Set<Map.Entry<String, Integer>> set;
BufferedReader input = new BufferedReader(new InputStreamReader(System.in));
BufferedWriter output = new BufferedWriter(new OutputStreamWriter(System.out));

while(!(line = input.readLine()).equals("****END_OF_INPUT****"))
{
map.clear();
count = 0;

if(!line.equals("****END_OF_TEXT****"))
{
do
if(line.length()>0)
{
words = line.split("[,.:;!?\"\\(\\) \\t]+");
count += words.length;

for(String s : words)
{
s = s.toLowerCase();
map.put(s, map.containsKey(s) ? map.get(s)+1 : 1);
}

}
while(!(line = input.readLine()).equals("****END_OF_TEXT****"));

lg = Math.log10(count);
E_T = 0;
set = map.entrySet();

for (Map.Entry<String, Integer> entry : set)
E_T += (entry.getValue() * (lg - Math.log10(entry.getValue())));

if(count>0)
E_T /= count;

output.write(String.format("%d %.1f %d\n", count, E_T, (int)(E_T*100/lg)));
}
else
output.write("0 0.0 0\n");
}

output.flush();
}
}``````
Do or do not. There is no try.

brianfry713
Guru
Posts: 5947
Joined: Thu Sep 01, 2011 9:09 am
Location: San Jose, CA, USA

### Re: 860 - Entropy Text Analyzer

input:

Code: Select all

``````word1 word2
\\
word3
****END_OF_TEXT****
****END_OF_INPUT****
``````
AC output:

Code: Select all

``````4 0.6 100
``````
Check input and AC output for thousands of problems on uDebug!

Scarecrow
Learning poster
Posts: 69
Joined: Wed Oct 19, 2011 9:06 pm

### Re: 860 - Entropy Text Analyzer

still WA. please help me find the bug in the code brianfry713.

Code: Select all

``````import java.io.*;
import java.util.*;

class Main
{
public static void main(String[] args) throws Exception
{
int count, i;
double lg, E_T;
String line, s, words[];
TreeMap<String, Integer> map = new TreeMap<String, Integer>();
Set<Map.Entry<String, Integer>> set;
BufferedReader input = new BufferedReader(new InputStreamReader(System.in));
BufferedWriter output = new BufferedWriter(new OutputStreamWriter(System.out));

while(!(line = input.readLine()).equals("****END_OF_INPUT****"))
{
map.clear();
count = 0;

if(!line.equals("****END_OF_TEXT****"))
{
do
{
words = line.split("[,.:;!?\"\\(\\) \\t]+");

if(words.length>0)
{
count += words.length;

if(words[0].equals(""))
{
i = 1;
count--;
}
else
i = 0;

for(; i<words.length; ++i)
{
s = words[i].toLowerCase();
map.put(s, map.containsKey(s) ? map.get(s)+1 : 1);
}

}
}while(!(line = input.readLine()).equals("****END_OF_TEXT****"));

lg = Math.log10(count);
E_T = 0;
set = map.entrySet();

for (Map.Entry<String, Integer> entry : set)
E_T += (entry.getValue() * (lg - Math.log10(entry.getValue())));
E_T /= count;

output.write(String.format("%d %.1f %d\n", count, E_T, (int)(E_T*100/lg)));
}
else
output.write("0 0.0 0\n");
}

output.flush();
}
}``````
Do or do not. There is no try.

brianfry713
Guru
Posts: 5947
Joined: Thu Sep 01, 2011 9:09 am
Location: San Jose, CA, USA

### Re: 860 - Entropy Text Analyzer

input:

Code: Select all

``````word1 word2
\\ \\
word3
****END_OF_TEXT****
****END_OF_INPUT****
``````
AC output:

Code: Select all

``````5 0.6 83
``````
Check input and AC output for thousands of problems on uDebug!

anacharsis
Learning poster
Posts: 69
Joined: Mon Feb 09, 2015 1:56 am

### Re: 860 - Entropy Text Analyzer

If you are going to do this in Java, make sure you set up your stream reader with the ISO-8859-1 charset.

Sample using a buffered reader:

Code: Select all

``````	private static BufferedReader br;

static String readln() throws IOException {
if ( br == null ) {
br = new BufferedReader( new InputStreamReader( System.in, "ISO-8859-1" ), 65536 );
}
return br.readLine();
}

``````

sith
Learning poster
Posts: 72
Joined: Sat May 19, 2012 7:46 pm

### Re: 860 - Entropy Text Analyzer

It is bad. How should I figure out the encoding requirement from problem description? It should be clear from the problem statement.