Using AI in Genealogy

Blog post banner Using AI for Genealogy

Thanks for such a warm reception at the Western New York Genealogical Society this past weekend. It was a pleasure to be talking about “Using AI for Genealogy” as part of their year-long fiftieth anniversary celebration, conducted over Zoom. At least fifteen states and two countries were represented in the audience.

The lecture was for people who have not already used AI tools but wanted to learn about them and how to start, AND for those who were already using the tools to share ideas about how to be more effective and expand their use.

It took over a day to obtain the ChatGPT data export that I mentioned during the lecture, but it did arrive later in the afternoon. As a reminder, this data export of all your chats can be requested by clicking on the profile icon on the lower left -> Settings -> Data Controls tab -> Export data. The link allows you to download a zipped file, and when you open it, use an HTML file to access your chats.

I wanted to share some of the great feedback from the audience:

  • “Fantastic ‘Gen AI 101’ and how to apply it to research!!!”
  • “Thank you so much! Very clear. Makes me want to go out and try it.!”
  • “Fantastic program!”
  • “This was perfectly demonstrated. Thank you!”
  • “Wow! So much information. Thank you so much.”
  • “I learned so much.  No longer afraid to try it.  Thank you.”
  • ” Hope I can find the time to watch this over and over and over!”

You can embark on a captivating exploration at the crossroads of genealogy and artificial intelligence with our lecture on “Using AI in Genealogy,” conducted over Zoom. Presented by a seasoned genealogist who holds a Ph.D. in Computer Science & Engineering and is the author of “Crash Course on ChatGPT and Genealogy ,” this session promises practical ways to get started using text-to-text artificial intelligence, prompt engineering and other AI tools, with some technological background. AI tools into your genealogical research, along with some technological background about generative AI.

The reasonable pricing ensures accessibility for your group, and participants will receive a thoughtfully curated 5+ page handout. Additionally, we’ve included some optional ‘Homework, but not to turn in’ for those who are ready to delve deeper into the subject. Contact us now to secure an engaging, informative, and educational Zoom lecture for your group.

The reasonable pricing ensures accessibility for your group, and participants will receive a thoughtfully curated 5+ page handout. Additionally, we’ve included some optional ‘Homework, but not to turn in’ for those who are ready to delve deeper into the subject. Contact us now to secure an engaging, informative, and educational Zoom lecture for your group.

There’s an AI for That: Transcribing Handwriting

Blog Post Banner There’s an AI for that transcribing text

Despite what you might have heard, there is progress being made on anything an AI can help with, including handwriting-to-text. In this blog post, we will cover just a few of the AI tools available for transcribing images of handwritten documents into text. The conversion can be done using digital images created by scanning or photographing handwritten documents.

Transcribing documents (or important parts of documents) is a thing that I always recommend. Reading a document is passive. The motion of writing or typing a document forces us to engage different parts of our brain with its content.

Even if a tool pulls the text out of an image, there is still work to be done in checking the accuracy and formatting the text.

While this can be done with a pencil and piece of paper, I always write the transcription into a word processing document. A word processing document is easier to share and extract the key pieces of data. Be sure to store the original image and the transcription together on your computer. 

NOTE: Always consider any privacy concerns before uploading documents to a website. While the website may not store the image, it may be used to train the AI model. Anything uploaded to a website usually travels through several stops on its way through the internet to the website and back. 

NOTE: The results from these experiments are certainly influenced by the quality of material that is input. This means that your results may vary.

I am not affiliated with any of the products mentioned in this review.

Always check usage rights for what is generated by a tool.

As the “Unofficial Historian for the 51st Pioneer Infantry Regiment,” we are always on the lookout for materials that add to the understanding of the Regiment’s service in World War I. We located some letters and decided to try out some handwriting-to-text AI tools.

The beginning of one of the letters was:

first part of WWI letter
horizontal-line

OCR2Edit

OCR2Edit has tools to extract text from scans, images and includes more  features. Since the tools are focused on text, and there was no explicit tool for converting handwriting into text, I had low expectations that this would be the right tool for the task.

At the time of writing this blog post, 3 tasks per hour could be done for free.

OCR2Edit homepage

I selected the Image to text tool and followed the directions to start the process.

When the tool was done, I could download the text file with the transcription.

OCR2Edit download page

The transcription of the letterhead was good, but the handwritten part was not helpful.

OCR2Edit results
horizontal-line

Aspose

Aspose OCR app is an online tool is for turning handwritten notes to text.

Aspse OCR App homepage

The interface on this webpage is slightly awkward.

The first page of a letter uploaded and the “Recognize” button clicked. Then it is time to wait. It took a while to process the request, but there was an option to bookmark the page and return to it.

There are buttons for several of their other Optical Character Recognition (OCR) apps that might be more useful.

The format for download was selectable from a drop-down menu.

Aspose format for download was selectable from a drop-down menu.

There was also an option to apply Automated Text correction.

The results are downloaded into a file named “results” which is less useful than a file that has the original filename in it.

There is a button for Options on the Home Page, where you can select: Enhance Contrast, Deskew Image and Upscale Resolution.

Aspose options

All of the options were selected in an attempt to get better results, but there was no improvement.

horizontal-line

Pen TO Print

Pen TO Print was the best tool in this set of experiments.

Only the first 10 pages are free, so check out the pricing if you need to do more.

Pen TO Print Homepage

Select Handwriting to Text Converter. Then Add Files, by dragging and dropping the file or clicking the plus to open a dialog box to navigate to and select a file (or files). Then select Convert.

Pen TO Print Add Files

The text can be Download as Text or Word document, or copied to the Clipboard. The filename of a downloaded files is the original filename with “Pen2Print-Export” added to it. This feature helps keep track of the transcribed files on your computer.

Pen TO Print Download

This was by far the best of the tools that were tried. The output needs some minor corrections, and formatting. Both of these tasks will engage the brain, and make us think about the content.

Let us know how you do!

Back to School: Genealogy Style

Blog Post Cover: Back to school genealogy style

When autumn comes, we think of going back to school. Genealogists are always learning, and webinars are a great way to do that. Presentations give us information, introduce us to new techniques or provide a new way of looking at our research. These resources in the blog post offer great classes and more.


The Genealogy Center at the Allen County Public Library hosts the Periodical Source Index (PERSI) and has many recorded webinars available on its YouTube Channel. You can even send them an email if you have a question.

ACPL Genealogy Center Home Page

The Midwest Genealogy Center at Mid-Continent Public Library offers a variety of resources. You can even request an Appointment with a Genealogy Consultant. Be sure to check out their upcoming and register for them on their events page. You can view their recorded talks on their YouTube Channel.

The Midwest Genealogy Center at Mid-Continent Public Library Home Page

BYU’s Harold B. Lee Library offers new webinars every week. They also offer a large library of recorded webinars.

BYU Library Webinar Page

Of course for the more adventurous, consider a class at your local community college.

When everyone around is going back to school, join them!

ChatGPT and GEDCOM Files

Blog Post Banner ChatGPT and GEDCOM Files

Before I was a professor, I was a flight test engineer. My love of testing systems goes back to my early days working in a lab during college. My particular gift was always find a way to “break” hardware or software through use. My desire to investigate the use of ChatGPT in genealogy has definitely coincided with my enjoyment of testing. In this blog post, I take a look at what ChatGPT knows about GEDCOMs, how it builds one and how it can create a narrative when given an individual’s data formatted in a GEDCOM.

The technical jargon in this paragraph is available for those who want a slightly deeper understanding. In computer science, data can be grouped together in meaningful representation of things that live in the real world. A data structure is a way to group fields in a specific order for a program to input data, manipulate it, and output it. The way that genealogical data is formatted and shared is the GEnealogical Data COMmunication (GEDCOM) standard.

GEDCOM (Genealogical Data Communication) is a file format used to exchange genealogical data between different genealogy software programs. It is a standard format for saving family tree data, and it allows users to transfer their family tree data from one program to another.

GEDCOM files are saved with the extension “.ged” and are made up of text-based data that includes information about individuals, families, and events such as births, marriages, and deaths. The data is organized in a hierarchical format, with each record containing information about a single individual or event.

GEDCOM files can be used to create family trees, research family history, and share information with other genealogists. They are widely used by genealogy software programs and online genealogy databases. For example, you can export a GEDCOM from your family tree program or download a GEDCOM from Ancestry.com.

NOTE: DO NOT ENTER PRIVATE OR SENSITIVE DATA INTO ChatGPT. Your data is used for training, and is reviewed by OpenAI to verify that content complies with their policies and safety requirements. They may be used for training purposes.

I asked ChatGPT what it knew about GEDCOMs with prompts: What is a GEDCOM file? What is the GEDCOM standard? What are the fields in the GEDCOM standard?

ChatGPT answered reasonably well, except that it confidently stated the latest version of GEDCOM being used was 5.5.1. This is understandable because ChatGPT’s training ended in 2021. (As of the writing of this blog post, the current version  is 7.0. For more information see the FamilySearch wiki entry for GEDCOM.)

Knowing that ChatGPT was using GEDCOM 5.5.1 was not a problem for these experiments.

Creating a GEDCOM

I would not choose to build a GEDCOM in this manner, but I could see how entering a narrative about ancestors into the prompt and let ChatGPT build the relationships from written language could be helpful. Beginning a family tree or adding a separate branch could be done by ChatGPT, then imported into a family tree program.

Investigating how effective ChatGPT was at creating a simple GEDCOM, I asked it to:

Create a GEDCOM file for James Charles McMahon, born 10 Oct 1920, father Joseph Francis McMahon, mother was Ella Small.

GEDCOM file from ChatGPT

ChatGPT extracted the information from my request and filled in the fields. I only asked for a simple GEDCOM file, and had been very specific in what details to include. ChatGPT did fine with this request. You can see the button to copy the code so that I could store it in a file with a .ged extension that would be usable by a family tree program that conformed to the GEDCOM specification. In fact, it even warned me:

ChatGPT warning

By the way, the clipboard next to the response lets a user copy the whole response so that a user can paste the response into the document of their choice. When clicked, the clipboard turns into a checkmark momentarily, then returns to being a clipboard. The thumbs up and thumbs down allow a user to provide additional feedback. If the feedback is thumbs down, another version of the reply is generated and a user has the opportunity to share whether the new one or previous response is better, or if they were the same. Giving feedback is always optional.

ChatGPT feedback

NOTE: This is a representation of an individual in a GEDCOM format and is not a file that can be directly imported into a family tree program. The header and footer information is not present, however, I could give ChatGPT that information and ask it to update the GEDCOM to include it.

I tried again with a new prompt that contained more details about the person’s life:

Create a GEDCOM file for James Charles McMahon, born 10 Oct 1920 in Brooklyn, Kings County, NY, father Joseph Francis McMahon, mother was Ella Small. James Charles McMahon died on 28 Nov 1987 in New York, New York, New York, US.

The response was filed the additional data correctly into the GEDCOM:

Updated GEDCOM file from ChatGPT

Using the GEDCOM as input to a family tree program

I asked for the file in a couple of different ways, but ChatGPT gave me only the section of the file for an individual. Rootsmagic had problems with importing this and creating a family tree, but after a little experimentation, I found that was because the was missing the header and trailer information. This was quickly remedied by editing the file.

It was interesting how the placeholder text for the birth and date information for the individual’s mother and father was inserted into the GEDCOM to be interpreted by the program. Of course, this could be fixed later in the conversation by asking for an updated GEDCOM with this information. As the chat went on, I also gave ChatGPT their marriage information and asked it to update the GEDCOM.

Creating a narrative from a GEDCOM

For my next experiment, I copied the second GEDCOM that ChatGPT had generated and fed it back into the prompt, asking:

Write a narrative for James Charles McMahon given his GEDCOM information:

0 @I1@ INDI

1 NAME James Charles /McMahon/

1 SEX M

[the rest of the file is not shown for brevity]

ChatGPT had learned details from our previous conversation, and inserted details about the individual learned from previous GEDCOMS. Starting the request in a new conversation brought its knowledge about the individual back to the nothing and the story included only the information from the prompt.

Of course, ChatGPT only uses what I told it. In reality, this individual was not an only child. Interestingly, after it writes that he grew up in a family of three, with himself and his parents, he was depicted as a beloved brother. This is due to large language models relying on their training to build the next part of their output.

Next, I checked if the format of the input mattered to ChatGPT, and made the GEDCOM data into one continuous stream, rather than distinct lines, in my prompt:

Write a narrative for James Charles McMahon given his GEDCOM information:

0 @I1@ INDI 1 NAME James Charles /McMahon/ 1 SEX M [the rest of the file is not shown for brevity]

ChatGPT did not need lines of the file to be formatted; it interpreted the data correctly then wrote a narrative. (This is also true when entering data from a table into the prompt.) Without information about the individual’s parents death, the model built the text that they survived him, and in the same sentence that they were deceased before his passing. ChatGPT can appear to loose its mind, so always proofread any output before using it.

Next, I carved out the lines for this individual from a GEDCOM that had been exported from a family tree program, complete with source citations embedded in the code. This text was used it as input to ChatGPT, and I asked it again to write a narrative from the GEDCOM. ChatGPT was successful in capturing the details it knew. It also created some generalizations like: “Throughout his life, James was a beloved member of his family and community.” It also added context without being prompted: “Though we don’t have much information about his specific experiences, we can imagine that he lived through many significant moments in history, including World War II and the civil rights movement.”

The tales that ChatGPT weaves from a user’s input can be a combination of technically accurate and fanciful. The facts that are input can be woven into a smoother and grammatically correct output. Any additional text that ChatGPT generates or additional contextual content it adds does need to be verified. ChatGPT is a generative language model that creates sentences without judgement, and those facts are presented as correct.  (Always check the details that ChatGPT adds, as it may “hallucinate”!)

ChatGPT generates text with an optimistic tone. The tales do all seem to end on a positive note, reminding me of appending “and a good time was had by all” to a story.

As with any tool, how we used the output matters. ChatGPT has the flexibility to regenerate a response to our prompt, and we have the ability to edit the text as we see fit. This tool could be helpful to a genealogist trying to get started on that family history they have been planning to write. ChatGPT can help someone get around a writer’s block by providing a starting place. It can also proofread what you generate. All you have to do is ask.

It was instructive to see how the narrative text that was put into the prompt was translated into lines in the GEDCOM file. I enjoyed peaking under the hood of the implementation that is at the heart of family tree programs.

Let me know how you do, and send along any questions.

ChatGPT May 3 Version was used for these experiments. Expect ChatGPT to change over time as the technology matures.

Please check out other posts about ChatGPT and Artificial Intelligence:

5 Ways to Use ChatGPT to Research an Ancestor

Getting Started with ChatGPT

Artificial Intelligence and Genealogy

Cultural Anthropology and Genealogy

Blog Header Cultural Anthropology and Genealogy

Cultural Anthropology

Last semester I took a third course in anthropology. After taking courses in Archaeology and Biological Anthropology, the next for me to tackle was Cultural Anthropology. (Our local community college does not offer a course in the fourth area of anthropology, linguistic anthropology.) Due to the nature of the subject material, this class was the least rooted in hard science. Cultural Anthropology studies how a society organizes itself. This is done through its beliefs, and how people live, think, create and find meaning. It introduces the concept that cultures have an intrinsic logic in their practices.

A big part of this branch of anthropology is fieldwork. Anthropologists in the field study societies, collecting data to build ethnographies. This data is often qualitative. Originally fieldworkers studied societies as impartial and distant observers; later they shifted to coming off the veranda to be participant observers.

When we go beyond our ancestors’ birth and death dates to fill in the dashes with what they did between those two dates, we are doing something similar to the fieldwork done by anthropologists. We often wish that we could go back in time to come off the veranda to be participant observers but lacking that option we can use the older anthropologists’ method of building their work on others’ first-hand source material. In our pursuit, we can use published sources that were contemporary to their times to learn about their culture at their time. When we research and write about our ancestors, we are building an ethnography. We can interact with the artifacts that they and their contemporaries left behind, which is like the activities of archaeologists.

Even though we cannot be participant observers in our ancestor’s society during their time, sometimes we can participate with a society that is close to theirs. This can be done through participating in ethnic crafts, cooking, dancing, clothing, reading the books they read, learning stories they told and heard, and learning about or practicing their beliefs.

Interview with Mark Hildebrand about the Annapolis Past Port Wiki

Blog Post - Annapolis Past Port

Recently we had a chance to speak with Mark Hildebrand, the Executive Director of Make Your Mark Media, Inc., in Annapolis, MD, to discuss a remarkable collaborative project that captures the memories and history of community members. In this interview you can learn about a creative and engaging approach to capturing history and how you can participate in this project.

What is the Annapolis Past Port Wiki?

Basically, Annapolis Past Port is a history wiki for stories and history in and around Annapolis, Maryland. It is free and open to the public, and it uses the same software and structure as Wikipedia. It was created in 2017 as part of a summer internship program by Make Your Mark Media – an Annapolis-based nonprofit. The interns were Science Technology Engineering and Mathematics (STEM) students enrolled in Anne Arundel County Public Schools. We wanted to capture stories about people, places, things and events as they are remembered by the community that experienced or heard about them. The focus of Annapolis Past Port is not the hard facts and statistics that are on Wikipedia already, but the memories, stories and even tall tales that should be preserved for future generations.

What motivated you to begin the Annapolis Past Port Wiki?

Several years ago, one of my board members asked me if there was a way to develop a public database of historic sites and people in the Annapolis area. She had attended a conference in New England where they showcased one developed by a local historic society. Because it was a series of web pages, it seemed a bit restrictive and reliant upon a web developer to create the content. I found that a wiki could be a great format that would make it easy for the public to upload content and share it. And unlike Facebook or Blogs, that content would not get buried under subsequent entries.

What challenges have you faced with the wiki and what surprises have you had?

One of the biggest challenges has been to get others to add content to the wiki. I have created most of the current pages, and although I have reached out to local historians and even done a few public workshops, few have taken the next step to upload their research or stories. So I was very pleasantly surprised when one of our interns from this past supper created wiki pages for a Nike missile site I had never heard of, just outside of Annapolis. And then one on Lee Airport in Edgewater. As with all of the wiki pages, they need more information and contributions from other sources, but they are a great beginning.

Where can people find out more about Past Port?

You can find the Annapolis Past Port history wiki at pastport.org. You can browse and search the wiki. The Main Page has a list of some of the recently added pages. There are images as well as audio and video. 

How can people participate in the Annapolis Past Port?

Anyone is welcome to create an account and add or edit content. We have provided links to guidelines on creating and formatting content. Through links to Make Your Mark Media (www.mym-media.org) you can contact me at mark@mym-media.org for direct assistance.