Despite what you might have heard, there is progress being made on anything an AI can help with, including handwriting-to-text. In this blog post, we will cover just a few of the AI tools available for transcribing images of handwritten documents into text. The conversion can be done using digital images created by scanning or photographing handwritten documents.
Transcribing documents (or important parts of documents) is a thing that I always recommend. Reading a document is passive. The motion of writing or typing a document forces us to engage different parts of our brain with its content.
Even if a tool pulls the text out of an image, there is still work to be done in checking the accuracy and formatting the text.
While this can be done with a pencil and piece of paper, I always write the transcription into a word processing document. A word processing document is easier to share and extract the key pieces of data. Be sure to store the original image and the transcription together on your computer.
NOTE: Always consider any privacy concerns before uploading documents to a website. While the website may not store the image, it may be used to train the AI model. Anything uploaded to a website usually travels through several stops on its way through the internet to the website and back.
NOTE: The results from these experiments are certainly influenced by the quality of material that is input. This means that your results may vary.
I am not affiliated with any of the products mentioned in this review.
Always check usage rights for what is generated by a tool.
As the “Unofficial Historian for the 51st Pioneer Infantry Regiment,” we are always on the lookout for materials that add to the understanding of the Regiment’s service in World War I. We located some letters and decided to try out some handwriting-to-text AI tools.
The beginning of one of the letters was:
OCR2Edit
OCR2Edit has tools to extract text from scans, images and includes more features. Since the tools are focused on text, and there was no explicit tool for converting handwriting into text, I had low expectations that this would be the right tool for the task.
At the time of writing this blog post, 3 tasks per hour could be done for free.
I selected the Image to text tool and followed the directions to start the process.
When the tool was done, I could download the text file with the transcription.
The transcription of the letterhead was good, but the handwritten part was not helpful.
Aspose
Aspose OCR app is an online tool is for turning handwritten notes to text.
The interface on this webpage is slightly awkward.
The first page of a letter uploaded and the “Recognize” button clicked. Then it is time to wait. It took a while to process the request, but there was an option to bookmark the page and return to it.
There are buttons for several of their other Optical Character Recognition (OCR) apps that might be more useful.
The format for download was selectable from a drop-down menu.
There was also an option to apply Automated Text correction.
The results are downloaded into a file named “results” which is less useful than a file that has the original filename in it.
There is a button for Options on the Home Page, where you can select: Enhance Contrast, Deskew Image and Upscale Resolution.
All of the options were selected in an attempt to get better results, but there was no improvement.
Pen TO Print
Pen TO Print was the best tool in this set of experiments.
Only the first 10 pages are free, so check out the pricing if you need to do more.
Select Handwriting to Text Converter. Then Add Files, by dragging and dropping the file or clicking the plus to open a dialog box to navigate to and select a file (or files). Then select Convert.
The text can be Download as Text or Word document, or copied to the Clipboard. The filename of a downloaded files is the original filename with “Pen2Print-Export” added to it. This feature helps keep track of the transcribed files on your computer.
This was by far the best of the tools that were tried. The output needs some minor corrections, and formatting. Both of these tasks will engage the brain, and make us think about the content.
With the explosion of artificial intelligence (AI) tools available, have you considered how many tasks in genealogy might be made easier with its use? A type of AI is used to make suggestions based on your previous purchases. Search experiences are beginning to incorporate AI. AI tools can be used in the creative process to invent images and content. AI is already incorporated into many tasks already done in genealogy, such as translation and indexing.
In upcoming blog posts, we will explore a few ways that genealogical tasks that can be done more efficiently with AI. We will look for ways AI might help with a task that has been looming or to overcome hurdles in your way.
Think about tasks that you do in genealogy that could be automated or done more efficiently by using AIs. There is probably already an AI tool for that. Some potential ideas are:
Extracting text from:
Images of typewritten text
Images of handwritten text
Converting interviews to text
Editing images to:
Remove backgrounds
Remove parts elements in the pictures
Working with text
Summarizing text
Analyzing text
Formatting text
Formatting citations
Writing content
NOTE: There are many other AIs that can do amazing things. Check out recommended tools from sources that you trust. While you can search the web for tools but be sure to trust your antivirus software and only click on safe websites. The best suggestion is to always try before paying any fee.
NOTE: Do not upload sensitive information into any AI tool.
NOTE: DO NOT ENTER PRIVATE OR SENSITIVE DATA INTO ChatGPT. Your data is used for training, and is reviewed by OpenAI to verify that content complies with their policies and safety requirements. Data may be used for training purposes.
ChatGPT has a new feature, and it’s pretty nice! Next to the name of the chat there are buttons to edit the name of the chat, share the chat and delete the chat.
Now you can share your chats via the click of a button.
When you share the chat, anyone with the link will be able to see what you and the AI chatted about, up until the time that you generated the link. If your conversation with the AI continues after the link has been created (by sending new messages to the AI), the new part of the conversation will not be shared.
When you click on the button to share the chat, you are able to see a preview of what will be shared.
Click on the three dots (sideways snowman) to share your username when the link is opened.
The other option is to share anonymously.
Click on the Copy Link button, and the link can then be pasted in an email, or by other means.
A really interesting feature is that the person with the link has the option to Continue this conversation if they are logged into their OpenAI account.
A user with the link can import your conversation into their own chat history. They will have that part of your conversation, even if you delete the conversation in your account.
You can delete the link, and it will no longer work.
In addition to sharing a chat, I found the shared link feature helpful for copying the text of a conversation and pasting it into a word processing document. I opened a new tab in my browser and pasted the link.
It is inevitable that similar AI tools will be compared. This blog post takes a look at comparing OpenAI’s ChatGPT and Google Bard.
When Google Bard was asked how it was different from ChatGPT, it answered that its training data contained images, that it could access the internet, and that it was a more general AI rather than a text-generating AI. Bard also told me that while ChatGPT was creative, it was more creative.
Google Bard has an interesting approach to answering prompts. Unlike ChatGPT, its training and knowledge does not end at 2021. It can go out and get content from the web to generate its answers.
Like ChatGPT, Google Bard can also hallucinate and authoritatively state inaccurate information. The “Google it” button found under a Bard response can help comfort you that the answer is not a hallucination.
These two AI tools do have some differences.
ChatGPT offers multiple conversations so that a user’s conversations can stay organized. It also has the ability to present previous conversations and pick up where it left off. Google Bard holds one conversation. It allows users to return to their previous prompts by selecting their Bard Activity from its menu. The responses to the prompts have to be selected at the time, and cannot be recalled.
Bard responses can contain images returned from the web. The responses, without any images, can be uploaded to a document in the user’s Google Drive or into the text of a gmail. ChatGPT responses are text-only, and need to be copied and pasted from the browser into a document. (Note: browser addons or plugins to capture responses are not discussed in this blog post. Any code you add to your browser this way should be researched thoroughly!)
Given that the content generated by Bard is not owned by the user, I will probably use ChatGPT for generating text and explaining concepts. (Of course, what ChatGPT generates should be verified!) I do prefer that responses are saved, and that different chats can be active.
It is possible that I might lean on Google Bard more for research questions, and will ask Bard for its sources. Undoubtedly the “Google it” button will be used in those efforts.
Of course, we can expect that Google Bard and other AI tools will continue to evolve at a rapid pace.
Based on the versions available at the time of this blog post being written, below is a table comparing features of OpenAI’s ChatGPT and Google Bard that I found notable.
Although Google Bard states that it removes personally identifiable information when using conversations to improve the model, DO NOT INCLUDE PERSONAL INFORMATION IN YOUR CHATS.
This week I spent some time working with Google’s challenger to OpenAI’s ChatGPT. Google Bard is a Language Model for Dialogue Applications (LaMDA), and I was working on the day that Bard began to bring images from Google Search into its results. Bard advertises that it helps you plan, solves complicated problems and supports your creative process. When it quotes content, it will cite where it found the material, or the computer code repository that was used.
In this article you will notice that I do include screenshots of answers, as I did for ChatGPT. That is because Google owns the generated content. If anyone wants to publish the content, they have to get permission from Google; at best Google and the user might share the rights to material. (In comparison, when you generate content in ChatGPT you own the rights to that content.)
You can select to save your Bard Activity, but even if that option is deselected, your activity is saved for up to 48 hours so that the feedback can be used. You can also select whether or not your activity is auto-deleted after 18 months. In my Bard Activity I could see the prompts that were given, but could not retrieve Bard’s responses, so remember to SAVE the responses. At the time of your session you can copy-and-paste them into a document or use the upload function described below.
The menu for Google Bard appears on the left hand side of the window.
At the bottom of each response were buttons to like, dislike, upload the response, or “Google it”
When you select to upload a response, you have options to upload it your Google Drive or draft an email in Gmail. The uploaded response will include text but NOT include the images you see in Bard.
I decided to ask some genealogically oriented questions, as I had done when testing ChatGPT.
“What are good resources for genealogy?” The answer to this prompt was a reasonable list of record databases, websites and societies.
“How can you help me with my genealogy?” This prompt was answered with ideas about genealogy research, finding resources, interpreting data and help create a family tree.
“How did someone travel from New York City to Newport, RI, in 1850?” Google Bard answered that the travel would have been by stage coach and steam ship, presenting me with images and data from the web along with its answer.
I asked what its sources were for this information, and it was listed sources. It also shared that it used its own knowledge to fill in the gaps.
When I asked how to find the source material, it provided links to places to buy a physical copy but did not provide a link to the source on Google Books. It did tell me that the book was available to view and download from Google Books. That resource was actually a very interesting discussion on travel to Newport between 1800 and 1850.
Since the user does not own the generated content, and the web is used to help answer prompts, this AI tool may be more useful to me as a research assistant.
Before I was a professor, I was a flight test engineer. My love of testing systems goes back to my early days working in a lab during college. My particular gift was always find a way to “break” hardware or software through use. My desire to investigate the use of ChatGPT in genealogy has definitely coincided with my enjoyment of testing. In this blog post, I take a look at what ChatGPT knows about GEDCOMs, how it builds one and how it can create a narrative when given an individual’s data formatted in a GEDCOM.
The technical jargon in this paragraph is available for those who want a slightly deeper understanding. In computer science, data can be grouped together in meaningful representation of things that live in the real world. A data structure is a way to group fields in a specific order for a program to input data, manipulate it, and output it. The way that genealogical data is formatted and shared is the GEnealogical Data COMmunication (GEDCOM) standard.
GEDCOM (Genealogical Data Communication) is a file format used to exchange genealogical data between different genealogy software programs. It is a standard format for saving family tree data, and it allows users to transfer their family tree data from one program to another.
GEDCOM files are saved with the extension “.ged” and are made up of text-based data that includes information about individuals, families, and events such as births, marriages, and deaths. The data is organized in a hierarchical format, with each record containing information about a single individual or event.
GEDCOM files can be used to create family trees, research family history, and share information with other genealogists. They are widely used by genealogy software programs and online genealogy databases. For example, you can export a GEDCOM from your family tree program or download a GEDCOM from Ancestry.com.
NOTE: DO NOT ENTER PRIVATE OR SENSITIVE DATA INTO ChatGPT. Your data is used for training, and is reviewed by OpenAI to verify that content complies with their policies and safety requirements. They may be used for training purposes.
I asked ChatGPT what it knew about GEDCOMs with prompts: What is a GEDCOM file? What is the GEDCOM standard? What are the fields in the GEDCOM standard?
ChatGPT answered reasonably well, except that it confidently stated the latest version of GEDCOM being used was 5.5.1. This is understandable because ChatGPT’s training ended in 2021. (As of the writing of this blog post, the current version is 7.0. For more information see the FamilySearch wiki entry for GEDCOM.)
Knowing that ChatGPT was using GEDCOM 5.5.1 was not a problem for these experiments.
Creating a GEDCOM
I would not choose to build a GEDCOM in this manner, but I could see how entering a narrative about ancestors into the prompt and let ChatGPT build the relationships from written language could be helpful. Beginning a family tree or adding a separate branch could be done by ChatGPT, then imported into a family tree program.
Investigating how effective ChatGPT was at creating a simple GEDCOM, I asked it to:
Create a GEDCOM file for James Charles McMahon, born 10 Oct 1920, father Joseph Francis McMahon, mother was Ella Small.
ChatGPT extracted the information from my request and filled in the fields. I only asked for a simple GEDCOM file, and had been very specific in what details to include. ChatGPT did fine with this request. You can see the button to copy the code so that I could store it in a file with a .ged extension that would be usable by a family tree program that conformed to the GEDCOM specification. In fact, it even warned me:
By the way, the clipboard next to the response lets a user copy the whole response so that a user can paste the response into the document of their choice. When clicked, the clipboard turns into a checkmark momentarily, then returns to being a clipboard. The thumbs up and thumbs down allow a user to provide additional feedback. If the feedback is thumbs down, another version of the reply is generated and a user has the opportunity to share whether the new one or previous response is better, or if they were the same. Giving feedback is always optional.
NOTE: This is a representation of an individual in a GEDCOM format and is not a file that can be directly imported into a family tree program. The header and footer information is not present, however, I could give ChatGPT that information and ask it to update the GEDCOM to include it.
I tried again with a new prompt that contained more details about the person’s life:
Create a GEDCOM file for James Charles McMahon, born 10 Oct 1920 in Brooklyn, Kings County, NY, father Joseph Francis McMahon, mother was Ella Small. James Charles McMahon died on 28 Nov 1987 in New York, New York, New York, US.
The response was filed the additional data correctly into the GEDCOM:
Using the GEDCOM as input to a family tree program
I asked for the file in a couple of different ways, but ChatGPT gave me only the section of the file for an individual. Rootsmagic had problems with importing this and creating a family tree, but after a little experimentation, I found that was because the was missing the header and trailer information. This was quickly remedied by editing the file.
It was interesting how the placeholder text for the birth and date information for the individual’s mother and father was inserted into the GEDCOM to be interpreted by the program. Of course, this could be fixed later in the conversation by asking for an updated GEDCOM with this information. As the chat went on, I also gave ChatGPT their marriage information and asked it to update the GEDCOM.
Creating a narrative from a GEDCOM
For my next experiment, I copied the second GEDCOM that ChatGPT had generated and fed it back into the prompt, asking:
Write a narrative for James Charles McMahon given his GEDCOM information:
0 @I1@ INDI
1 NAME James Charles /McMahon/
1 SEX M
[the rest of the file is not shown for brevity]
ChatGPT had learned details from our previous conversation, and inserted details about the individual learned from previous GEDCOMS. Starting the request in a new conversation brought its knowledge about the individual back to the nothing and the story included only the information from the prompt.
Of course, ChatGPT only uses what I told it. In reality, this individual was not an only child. Interestingly, after it writes that he grew up in a family of three, with himself and his parents, he was depicted as a beloved brother. This is due to large language models relying on their training to build the next part of their output.
Next, I checked if the format of the input mattered to ChatGPT, and made the GEDCOM data into one continuous stream, rather than distinct lines, in my prompt:
Write a narrative for James Charles McMahon given his GEDCOM information:
0 @I1@ INDI 1 NAME James Charles /McMahon/ 1 SEX M [the rest of the file is not shown for brevity]
ChatGPT did not need lines of the file to be formatted; it interpreted the data correctly then wrote a narrative. (This is also true when entering data from a table into the prompt.) Without information about the individual’s parents death, the model built the text that they survived him, and in the same sentence that they were deceased before his passing. ChatGPT can appear to loose its mind, so always proofread any output before using it.
Next, I carved out the lines for this individual from a GEDCOM that had been exported from a family tree program, complete with source citations embedded in the code. This text was used it as input to ChatGPT, and I asked it again to write a narrative from the GEDCOM. ChatGPT was successful in capturing the details it knew. It also created some generalizations like: “Throughout his life, James was a beloved member of his family and community.” It also added context without being prompted: “Though we don’t have much information about his specific experiences, we can imagine that he lived through many significant moments in history, including World War II and the civil rights movement.”
The tales that ChatGPT weaves from a user’s input can be a combination of technically accurate and fanciful. The facts that are input can be woven into a smoother and grammatically correct output. Any additional text that ChatGPT generates or additional contextual content it adds does need to be verified. ChatGPT is a generative language model that creates sentences without judgement, and those facts are presented as correct. (Always check the details that ChatGPT adds, as it may “hallucinate”!)
ChatGPT generates text with an optimistic tone. The tales do all seem to end on a positive note, reminding me of appending “and a good time was had by all” to a story.
As with any tool, how we used the output matters. ChatGPT has the flexibility to regenerate a response to our prompt, and we have the ability to edit the text as we see fit. This tool could be helpful to a genealogist trying to get started on that family history they have been planning to write. ChatGPT can help someone get around a writer’s block by providing a starting place. It can also proofread what you generate. All you have to do is ask.
It was instructive to see how the narrative text that was put into the prompt was translated into lines in the GEDCOM file. I enjoyed peaking under the hood of the implementation that is at the heart of family tree programs.
Let me know how you do, and send along any questions.
ChatGPT May 3 Version was used for these experiments. Expect ChatGPT to change over time as the technology matures.
Please check out other posts about ChatGPT and Artificial Intelligence:
