Skip to the content.

The function load_lemma takes a single argument and returns a dictionary consisting of {“word” : “lemma”} pairs.

  1. lemma_file is a string. It should be the name of your lemma list (don’t forget place this file in the same folder as your Python script and set your working directory!)
def load_lemma(lemma_file): #this is how we load a lemma_list
	lemma_dict = {} #empty dictionary for {token : lemma} key : value pairs
	lemma_list = open(lemma_file).read() #open lemma_list
	lemma_list = lemma_list.replace("\t->","") #replace marker, if it exists
	lemma_list = lemma_list.split("\n") #split on newline characters
	for line in lemma_list: #iterate through each line
		tokens = line.split("\t") #split each line into tokens
		if len(tokens) <= 2: #if there are only two items in the token list, skip the item (this fixed some problems with the antconc list)
		lemma = tokens[0] #the lemma is the first item on the list
		for token in tokens[1:]: #iterate through every token, starting with the second one
			if token in lemma_dict:#if the token has already been assigned a lemma - this solved some problems in the antconc list
				lemma_dict[token] = lemma #make the key the word, and the lemma the value


The example below uses Laurence Anthony’s lemma list, which includes lemmas for all words that occur at least twice in the BNC.

lemma_dict = load_lemma("antbnc_lemmas_ver_003.txt")