__How to Build Eliza Chatterbot - A Program that can Chat with Humans

				

How to Build Eliza
Chatterbot


A Program that can Chat with Humans



Written by Amit
Mathur on 10th to 12th Dec, 2002


What is Eliza ? 


Eliza is an AI Program that
simulates the behavior of a therapist. The first program of this sort was
developed in 1967 in MIT. Such programs, which interact with user in simple
English language and can simulate a conversation are known as Chatterbot. 


A program like Eliza requires
knowledge of three domains:

 
1.   
 Artificial Intelligence

 2.   
 Expert System

 3.   
 Natural Language Processing

Even though last two are
sub-parts of the first one, they are emerging as  science in themselves.


Eliza can not, of course,
think on its own. It has a repository or database of facts and rules, which are
searched to give the best possible response. 


Eliza works by matching
process.  Very rarely an entire
sentence is matched to give the response. 


The rules are indexed by
keywords. Some rules require no keyword.


How does it work ? 


Ninety
percent of what Eliza says is found in the associated Data File. This file acts
as Knowledge Base for the complete system. 


The
in-built responses comprise the Static Database of the system. These are
the responses for the following cases: 


1.   
When Eliza does not understand what the user is talking about.


2.   
When the user repeats himself.


3.   
When the user does not type anything and just keeps on pressing Enter.


4.   
For the greeting statements. 


The
following strategy is used to respond to a request: 


Step 1:
Eliza finds
out if the user has given any null input. If so, it takes the fact from the
static database to respond. 


Step 2:
There are
some in built responses that Eliza can recognize readily. It finds the presence
of any such sentence after fragmenting the user’s input and remembers the
associated keyword. This keyword defines the Context of the talk. 


Step 3:
If no
in-built sentence frame work is found, then the Eliza searches for the specific
keyword to define the context. If no context is found, it deliberately
motivates the user to speak about a specific topic.  


Step 4:
A response is
chosen (at this time, randomly) from the database of available responses. 


Step 5:
Any necessary
transpositions are done. For example, consider the following
conversation: 



 
SAM>
 I PLAN TO GO TO JAIPUR TOMORROW WITH MY WIFE.
 
 
 
ELIZA> AND
 WHAT HAPPENS IF YOU WON'T GO TO JAIPUR WITH YOUR WIFE ?

  
 
 

Here, the
word My has to be transposed to YOUR.


Step 6:
To simulate
the human conversationalists, Eliza simulates Typing and does so slowly with
making spelling mistakes and correcting them.

 


What
about Coding ?
Now let
us start the real coding portion. I am using Turbo C IDE 3.0 as this is
the IDE that most Indian Students use. 
Note
that the complete source code is in the Zip file that accompanies this file. But
my main stress is on approach and not on coding. The code,  which is
written in just 90 minutes, is good as a working skeleton. 
Before
going into the detailed coding aspect, let us first see the structure of a
sample Data File. Eliza recognizes certain keywords. If these keywords are found
in the user input, then corresponding to that, from a predefined set of
responses, one is chosen and displayed.
A keyword
is separated in the data file (called Dictionary) from the responses by @KWD@
token. This token indicates that the next line that follows is actually a
keyword, not a response.


 


 



@KWD@

HELLO

HI, HOW ARE YOU

HELLO DEAR !

@KWD@

I WILL

YOU WILL DO SO. I BELIEVE IT TOO...

WILL YOU BE ABLE TO DO SO ?

@KWD@

YES

ARE YOU SURE ?

HOW CAN YOU BE SO SURE ?

YOU SEEM TO BE VERY OPTIMISTIC.

@KWD@

NO

YOU SEEM TO BE VERY PESSIMISTIC.

NEVER SAY NO...

@KWD@

COMPUTER

I KNOW HOW TO WORK ON COMPUTER.

YOU ARE CURRENTLY USING A COMPUTER. RIGHT ?
<< Add whatever you want in above format >>
Contents
of file : Eliza.dat
 

 


For example, in response to
'Hello', from the above dictionary, Eliza will give one of the following
responses:

 

HI, HOW ARE YOU
 

HELLO DEAR !

Once this thing is
clear, let us now define the Data Structures that we will be using. We create
two classes : 

 

progstr - This is
used to store the user's input related information.
 

resp - This is used
to store the information about the various responses.

Let me give the code
first and then I will explain it.


 


 



class progstr

{

public:

char userip[MAX_USER_INPUT];

char keyword[30];

int keyfound;

int keyno;

int nullip;

    // constructor

    progstr()

    {

        keyno=-1;

        nullip=0;

        keyfound=0;

    }

}ip;



class resp

{

int tot_resp;

int last_resp;

char replys[MAX_RESP_NO][MAX_RESP_LEN];

char word[MAX_KWD_LEN];

public:

    // constructor

    resp()

    {

    tot_resp=0;

    last_resp=-1;

    }



    int getcount()

    {

    return last_resp;

    }



    void addword(char str[MAX_KWD_LEN])

    {

    strcpy(word,str);

    }



    char * getword()

    {

    return word;

    }



    void addresp(char str[MAX_RESP_LEN])

    {

    strcpy(replys[++last_resp],str);

    }



    // defined later

    void display_resp(int num);

    void quit_display_resp(int num);



};


Basic
Data Structures Involved
 

 


The
character array userip is used to store the line typed by the user.
Another array keyword is used to store the keyword, if any, found in that
input. If a keyword is found, we make int keyfound to 1 else, it remains
0, as it is initialized to 0 in the Constructor. keyno stores the key
number of the corresponding keyword.
nullip
indicates whether the user has given any Null input ie, he is just pressing
enter and doing nothing else.
Now
let us come to the second class, resp. The first data member, tot_resp
indicates the total number of responses for a given keyword. For example, for
the keyword Hello, we have 2 responses (see Eliza.Dat). So, for
that, tot_resp holds a value of 2. last_resp is used for the
function processing and its use will be clear later on. 
The
replies are actually stored in replys[MAX_RESP_NO][MAX_RESP_LEN]
and the corresponding keyword is stored in the array word. 
Description of Functions
in Class resp :

 

Constructor: 

This is used to initialize the total number of responses to 0. Why
last_resp is initialized to -1 will be clear when you look at the function
add_resp.
 

int getcount():

This function is used to get a count of how many responses are there for
a given keyword. 
 

 void addword(char
str[MAX_KWD_LEN]):

This is used to add a keyword.
 

char * getword():

Used to return the keyword for a particular object of class resp.
 

void addresp(...):

This is used to add a response corresponding to a given keyword.
 

void display_resp(int):

This is used to display the response to the user corresponding to a
given index number for the responses.  (actually it does more than that
!).
 

void
quit_display_resp(int):

Difference between this function and above function is that it is used
in the end when the user is quitting. So, it does not return the prompt to
the user.

Let us now create a function
that reads the contents of file Eliza.Dat in an array of objects of class resp,
which we name keys. Since I have already explained both - the format of .Dat
file and data structures, this function should by clear with little or no
effort. The code is commented wherever necessary.
 


 


 



void read_from_file()

{

ifstream fin;

int index=-1;

fin.open("eliza.dat");

char line[MAX_RESP_LEN];

while(fin)

{

  fin.getline(line,MAX_RESP_LEN);

  char *ptr=NULL;

  ptr=strstr("@KWD@",line);

  if(strlen(line)<1)

   {

   break;

   }

  else if(ptr!=NULL)

   {

   // the next line is a keyword

   fin.getline(line,MAX_RESP_LEN);

   keys[++index].addword(line);

   }

  else

  {

  // it is a response

  keys[index].addresp(line);

  }

} // end of while

} // end of function


Read
Contents of Eliza.Dat File
 

 


Now let us create a function
for global initialization of the transposition words. This function is easy and
I will not belabor it.




 


 



void initialize_global()

{

strcpy(wordin[0],"ARE");

strcpy(wordout[0],"AM");



strcpy(wordin[1],"AM");

strcpy(wordout[1],"ARE");



strcpy(wordin[2],"WERE");

strcpy(wordout[2],"WAS");



strcpy(wordin[3],"WAS");

strcpy(wordout[3],"WERE");



strcpy(wordin[4],"YOU");

strcpy(wordout[4],"ME");



strcpy(wordin[5]," I ");

strcpy(wordout[5],"YOU");



strcpy(wordin[6],"YOUR");

strcpy(wordout[6],"MY");



strcpy(wordin[7],"MY");

strcpy(wordout[7],"YOUR");



strcpy(wordin[8],"I'VE");

strcpy(wordout[8],"YOU'VE");



strcpy(wordin[9],"YOU'VE");

strcpy(wordout[9],"I'VE");



strcpy(wordin[10],"I'M");

strcpy(wordout[10],"YOU'RE");



strcpy(wordin[11],"YOU'RE");

strcpy(wordout[11],"I'M");



strcpy(wordin[12],"ME");

strcpy(wordout[12],"YOU");



strcpy(wordin[13],"YOU");

strcpy(wordout[13],"ME");

}


Basic
Transformations
 

 




Let us now write a function for displaying the
responses to the user. The first if statement in the for loop is used to make a
deliberate typing error to make it appear more human like ;-). One character is
randomly chosen for typing error. Special cases like New Line and Backspace are
separately considered. (Think why ?). Now I introduce something new. A special
character - *. Char * represents all of the text found AFTER the identified keyword,
and before one of the following punctuation marks. 



For example, consider the user input



AMIT > CAN I GO TO INDORE TOMORROW ? 

ELIZA > WHAT IF YOU DO NOT GO TO INDORE TOMORROW ? 



The underlined portion is not stored in the dictionary, rather it is taken from
the user input. In the file Eliza.Dat, we store this information as 



CAN I

WHAT IF YOU DO NOT *



Star (*) asks the program to simply copy whatever is typed after the keyword
(here CAN I ) in the user input, as it is. I hope that now the function of * as
a special keyword is clear. So, let us consider a more complicated case.



AMIT > CAN I GO TO INDORE TOMORROW WITH MY FRIEND ? 

ELIZA > WHAT IF YOU DO NOT GO TO INDORE TOMORROW WITH MY FRIEND? 



Obviously this is not what we wanted. I am supposed to go with my friend, not
one of Eliza ! So, we must perform some transformation also. When we think of
transformation, the sentence gets divided in the following 3 sections:

 

Text Before
Transposition Word. (here, GO TO INDORE TOMORROW WITH )
 

The Transposed
keyword. (here, YOUR, in place of MY)
 

Text After
Transposition Keyword. (here, FRIEND ? )

The following code tackles
the three cases in a very lucid manner.




 


 


void resp :: display_resp(int num)

	{

	cout<<"ELIZA > ";

	for(int i=0;i<strlen(replys[num]);i++)

	{

     // for deliberate typing errors 

     // (for simulating the human behavoir ;-)



     if(RanNum(6)==0)

     { 

     char c=RanNum(100);

     if(c=='\n' || c=='\b' || c==13)

          cout<<"w";

     else

          cout<<c;

     delay(RanNum(DELAY));

     // correcting the deliberate typing error

     cout<<"\b";

		}





// * is used to write anything after the keyword

// as it is, but with some transformations like 

// converting MY to YOUR.

if(replys[num][i]=='*')

		{

     char * s1=ip.userip+strlen(ip.keyword);

     short int flag=0;

     for(int m=0;m<TRANSPOSE;m++)

          {

          char * s2=wordin[m];

          char *ptr=NULL;

          ptr=strstr(s1,s2);



          if(ptr!=NULL)

          {

          // transposition word found in the

          // user input

          flag=1;



          // printing text before wordin[m]

          int times=ptr-s1;

          for(int i=0;i<times;i++)

          {

          delay(DELAY);

          cout<<ip.userip[strlen(ip.keyword)+i];

          }



          // printing the wordout

          cout<<wordout[m];

          // printing the left overs

          char c;

          c=*(ptr+strlen(wordin[m]));

          int t=0;

          while(c!='\0')

          {

          cout<<*(ptr+strlen(wordin[m])+t);

          t++;

          c=*(ptr+strlen(wordin[m])+t);

          }

          }

          } // end of for



     // if flag is still zero , this means no need
for 

     // transposing any word.

     if(0==flag)

     {

          char c;

          c=*(s1+strlen(ip.keyword));

          int t=0;

          while(c!='\0')

          {

          cout<<*(s1+t);

          t++;

          c=*(s1+t);

     }

			} // end of if

			break;

		}

	else

			{

				cout<<replys[num][i];

				delay(RanNum(DELAY));

			}

	} // end of for



// giving the prompt back to user

	cout<<"\n"<<user<<" > ";

	}


Function
for displaying Eliza's Response
 

 


Finally we can work out a
procedure for searching the keyword in the user's input.  MAX_KEY indicates
the number of keywords in the DAT file. Here we are simply searching whether the
keyword exists in the user input (anywhere).




 


 



void find_keyword()

{

int len=0;

int lenkey=0;

int key_no=0;

char teststr[50];

while((ip.keyfound==0) &&(key_no!=MAX_KEY))

{

// getting the length of the keyword

lenkey=strlen(keys[key_no].getword());



char *ptr=NULL;

ptr=strstr(ip.userip,keys[key_no].getword());

if (ptr!=NULL)

	{

	// keyword found !

	ip.keyfound=1;

	ip.keyno=key_no;

	strcpy(ip.keyword,keys[key_no].getword());

	break;

	}

key_no++;

}

}




Simple
Search Routine.
 

 




When all these routines are made, we integrate them in the main function. For
complete source code, please download the accompanying Zip file with the
article.


Future Prospects: 


Like other
AI programs, this program also has immense possibilities of improvement. 


The
following are the improvements, which can be made in it. 



 

Learning
by Time or Experience can be Implemented.
 

All the
previous talking can be stored in a array of strings, so that in case of
user contradicting himself/herself, ELIZA can contradict him.
 

A database
or a flat file, at least, can be used for the data and talk storage.
 

Sessions
and User-Password Pairs can be established, so that, even after the
completion of one session, next time, whenever the user enters his User Name
and Password, ELIZA, will get all the relevant data and previous talks,
related to the user, from the database itself.
 

A prospect
for the cache Memory can be made, so as make the retrieval of
data and information can be faster.  I
had written a Research Paper on this topic one year ago, Intelligent
Information Retriever - Inception of Artificial Intelligence in Search
Engines. Follow the link to download it if you are interested.
 

Standard
Search algorithm can be used for the faster or better search of the relevant
result or answer.
 

Various
Graphical signs and/or symbols can be incorporated to show emotions, making
the conversations more lively and more realistic in nature.

Important
Note About Sample Code:
Please note that the current functionality and
features of this program are very limited and they are just for accompanying the article. If you want
to make this program more intelligent, make entries in Eliza.Dat file.
You can also increase the string manipulation power of the program,
like considering multiple lines from the user, etc. I had written this code in 1 1/2 hr.
just to make it more easier for the readers of my article about what is
happening.
HOW SMART YOU MAKE YOUR ELIZA DEPENDS ON HOW FAR YOU EXTEND
THIS PROGRAM. THERE IS PRACTICALLY NO LIMIT ! 

THIS CODE IS THE MINIMAL WORKING SKELETON !!

Don't
forget to read README.TXT before making conclusions about the program !
Complete
Source Code:



Included in the accompanying Zip File.