Announcement

Collapse
No announcement yet.

Reunion and Ancestry.com playing nice

Collapse
This topic is closed.
X
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Reunion and Ancestry.com playing nice

    Sorry but this turned out to be a long one.

    I'm trying to get Reunion to play nice with Ancestry.com. The other way around is unlikely as Ancestry.com obviously has a vested interest in not wanting you to use anything else.

    I've been using Reunion as the primary tool for managing the data. The bulk of my side of the family research is done by hand. Being the first generation in the US, most, if not all records from my side of the family are not to be found on Ancestry.com at all. I've been manually going through records throughout Europe to find anything, most of it lost during WWII. On my wife's side however, Ancestry.com is an excellent research tool. Some lines of her family have been in this country for as long as 11 generations and that's exactly what Ancestry.com is good for.

    With that said, getting data back and forth is close to impossible. Getting data from Ancestry is somewhat OK. I load the GEDCOM file using the standard procedure (marking all, import, compare, merge, etc.) Getting the data out of Reunion to replace the tree in Ancestry.com simply does not work. If you select standard GEDCOM 5.5, it preserves the _APID records but messes up everything else (well, Reunion exports it correctly, but Ancestry.com gets lost when it can't find its own record idiosyncrasies). If I select an "Ancestry.com" type of GECOM for export, it drops all _APID records altogether or turn them into NOTE records.

    When importing a GEDCOM from Ancestry, Reunion will make all sources "Free Form". What I did was to create a new source type (I called it Ancestry.com) and added all the fields present in the Ancestry GEDCOM file, including the _APID and REPO field. I then convert one by one by hand from Free Form to this new Ancestry.com source type. That was an attempt to see if I could get it out in a way Ancestry.com would understand. That didn't make any difference.

    The example below is just a snippet from one single individual showing one single level 1 tag (RESI). It doesn't matter what level 1 tag I use, they all show the same behavior. The Reunion GEDCOM is of type "Ancestry.com". The first thing you notice is that the level 3 _APID for a level 2 SOUR record is dropped altogether. The second thing is that the level 0 SOUR record is completely different.

    I'm including a screen grab showing how I have Reunion's SOUR @64@ recorded.

    Is there a way to make this work?

    Thanks!

    Ancestry:
    1 RESI Age: 51Marital Status: Married; Relation to Head of House: Head
    2 DATE 1930
    2 PLAC Winston-Salem, Forsyth County, North Carolina, USA
    2 SOUR @S-713549403@
    3 PAGE Year: 1930; Census Place: Winston-Salem, Forsyth, North Carolina; Roll: 1690; Page: 2A; Enumeration District: 57; Image: 298.0; FHL microfilm: 2341424
    3 _APID 1,6224::76982193

    Reunion:
    1 RESI Age: 51Marital Status: Married; Relation to Head of House: Head
    2 DATE 1930
    2 PLAC Winston-Salem, Forsyth County, North Carolina, USA
    2 SOUR @S64@
    3 PAGE Year: 1930; Census Place: Winston-Salem, Forsyth, North Carolina; Roll: 1690; Page: 2A; Enumeration District: 57; Image: 298.0; FHL microfilm: 2341424

    Ancestry:
    0 @S-713549403@ SOUR
    1 REPO @R-1148721728@
    1 TITL 1930 United States Federal Census
    1 AUTH Ancestry.com
    1 PUBL Ancestry.com Operations Inc
    1 _APID 1,6224::0

    Reunion:
    0 @S64@ SOUR
    1 TYPE Ancestry.com
    1 NOTE Ancestry.com
    1 TITL 1930 United States Federal Census
    1 AUTH Ancestry.com
    1 PUBL Ancestry.com Operations Inc
    1 NOTE 1,6224::0
    1 TEXT 1,6224::76441536
    2 CONT
    2 CONT 1,6224::77041883
    < Repeats ad infinitum >
    Attached Files

    #2
    Re: Reunion and Ancestry.com playing nice

    Hi grubba,

    Yes it is possible to get this to work and I wrote an thread afew weeks back around APID tags and Ancestry, with some screen shot around how to do this

    Discuss things you use with Reunion: a Mac, word processor, etc.


    In summary, I use Ancestry and Reunion together, saving the APID information in Ancestry at a source and citation detail level. The latter is important and also shows why you have (as I do) entries at the citation level showing _APID 1,6224::76982193 and for the source relating to the citations, an entry of _APID 1,6224::0.

    The APID entry needs to be saved at the citation detail field for each reference of the source. But the entry should show the header record number, e.g. 76982193. Each record type, e.g. 6224 being the Ancestry database entry for all "1930 United States Federal Census" entries. If other words, another person living with that person, e.g. spouse, would have a different header record number but everything else from a _APID tag point of view, would be the same.

    The entry;
    "PAGE Year: 1930; Census Place: Winston-Salem, Forsyth, North Carolina; Roll: 1690; Page: 2A; Enumeration District: 57; Image: 298.0; FHL microfilm: 2341424
    3 _APID 1,6224::76982193"
    should be keyed into the detail field for the citation. I would key this as
    "PAGE Year: 1930; Census Place: Winston-Salem, Forsyth, North Carolina; Roll: 1690; Page: 2A; Enumeration District: 57; Image: 298.0; FHL microfilm: 2341424_APID 1,6224::76982193", basically, concatenating the two entries.

    When you export out of Ancestry, choose Ancestry rather than GEDCOM 5.5 but you need to convert split the APID entry from the NOTE tag. How? Use Applescript of a text editor like TextWrangler to search and replace _APID entries with a newline and _APID (refer Applescropt image). To get a newline character, hold the control key and press Enter key.

    Below are details from prior thread...http://reuniontalk.com/showthread.php?t=8876
    ---
    The APID tags are found in the GEDCOM file for Ancestry sources that have been saved in your tree. In Ancestry, click on Manage Tree and there is an Export Tree button to click (it takes afew seconds to generate and then you have an option to download the file). This saves a GEDCOM file (text file) which when opened contains the APID tags for each citation (i.e. search for _APID).

    Ancestry to my knowledge use these tags internally in their web site and FTM software but the GEDCOM file you export, will show these (provided you have saved Ancestry sources).

    For example, a residence event would look like this (some date and some address and source# would be specific to your research entry. The APID details would be specific to the Ancestry record for this research.

    1 RESI
    2 DATE Some date
    2 PLAC Some address
    2 SOUR @Source#@
    3 PAGE Some date
    3 _APID 1,1836::21884408

    The format is 1, followed by a APID database tag (sounds complicated but each record collection in Ancestry has a separate tag ID number followed by 2 colons, followed by the header tag for your record. When you look at the URL for each Ancestry record, there is a long text but it has a h=, the number that follows is the header.

    This information is keyed into Reunion. Once you know the Ancestry tag for one type of records, e.g. UK census 1911, you can use this for all 1911 census citation details with only the header part changing for each. The example above is the tag ID for NZ electoral roll information.

    In the above example, the citation detail you would enter in Reunion would be 1969_APID 1,1836::21884413 and an applescript like the following, converts this to two lines (*).


    (*
    Change Ancestry tags to a _APID GEDCOM tag
    *)
    replace "_APID" using "
3 _APID" searching in text 1 of text document "whatever your GEDCOM filename is" options {search mode:literal, starting at top:true, wrap around:false, backwards:false, case sensitive:false, match words:false, extend selection:false}


    To use Applescript with a GEDCOM, use a text editor like TextWrangler since it is scriptable. (*) The 3 _APID actually has a newline character (doesn't show up when I cut in pasted it in this reply, see photo of AppleScript below)

    The end result is that when you export a GEDCOM from Reunion, run the applescript to split the _APID tags to a separate line in the file, upload this converted GEDCOM file back to Ancestry. Each of your citation details should show an Ancestry icon next to it to show that the citation points to an Ancestry source record. Clicking on the citation, would show the detail with an option to click on the Ancestry source.

    It is an involved process taking abit of time. The attached photos show the end result in Ancestry and also, what the Reunion citation detail looks like for the example above. I use the same citation detail as Ancestry uses in its source citation, e.g. in the example, this is the year of the electoral roll.
    ---


    If you have any problems, let me know. I am happy to help - the process is abit involved but I have been using it for about 6 months or so now, converting all my source citations and sources that reference Ancestry to include _APID tags.

    Regards,
    PC
    Attached Files

    Comment


      #3
      Re: Reunion and Ancestry.com playing nice

      Sorry to take a while to respond and thank you for taking the time to explain what you've done. I had noticed your earlier post after searching for this all over. Your solution only solves one of the many problems with the exported GEDCOM file from Reunion when you want to upload to Ancestry.com.

      There are several issues with sources. I've written a Python script that deals with many of the issues but have yet to tackle the third, nightmarish one described below. In the process, I'm adding Reunion's _UID lines as a PAGE line to newly created Reunion source entries to the exported GEDCOM file so Ancestry will preserve it. That makes re-importing it simpler but I still have quite a bit of work to get that all working. It would be a heck lot easier if I had some kind of access to Reunion's database directly (a dream if it was some flavor of SQL). That way I would just write my own exporter and forget this GEDCOM conversion.

      The basic issue is when the _APID within a source in the individual record gets renamed to PAGE. Renaming it alone does not solve the problem. You must also fix the level 0 SOUR record it points to so its _APID is renamed from NOTE and replace the REPO that went missing altogether.

      How Reunion exports:

      Individual:
      2 SOUR @S55@
      3 PAGE 1,1788::412665748

      Source:
      0 @S55@ SOUR
      1 TITL U.S. Public Records Index, Volume 1
      1 AUTH Ancestry.com
      1 PUBL Ancestry.com Operations, Inc.
      1 NOTE 1,1788::0
      1 NOTE Ancestry.com

      What Ancestry expects:

      Individual:
      2 SOUR @S-713550627@
      3 _APID 1,1788::412665748

      Source:
      0 @S-713550627@ SOUR
      1 REPO @R-1148721728@
      1 TITL U.S. Public Records Index, Volume 1
      1 AUTH Ancestry.com
      1 PUBL Ancestry.com Operations, Inc.
      1 _APID 1,1788::0

      The next more complex issue is when Reunion removes all _APID lines from the individual records and concatenates them all into one long TEXT line in the source record. In this case, you have to build a list of those _APIDs and map them to each source line in the individual records.

      How Reunion exports:

      Individual:
      2 SOUR @S56@
      3 PAGE Birthdate: 20 Apr 1989; Birth County: El Dorado

      Source:
      0 @S56@ SOUR
      1 TITL California Birth Index, 1905-1995
      1 AUTH Ancestry.com
      1 PUBL Ancestry.com Operations Inc
      1 NOTE 1,5247::0
      1 NOTE Ancestry.com
      1 TEXT 1,5247::19893513
      2 CONT
      2 CONT 1,5247::18624098
      2 CONT
      2 CONT 1,5247::6985158
      <

      Comment


        #4
        Re: Reunion and Ancestry.com playing nice

        Originally posted by grubba View Post
        ...Your solution only solves one of the many problems with the exported GEDCOM file from Reunion when you want to upload to Ancestry.com....There are several issues with sources. I've written a Python script that deals with many of the issues but have yet to tackle the third, nightmarish one described below....I'm adding Reunion's _UID lines as a PAGE line to newly created Reunion source entries to the exported GEDCOM file so Ancestry will preserve it. ...The basic issue is when the _APID within a source in the individual record gets renamed to PAGE. Renaming it alone does not solve the problem. You must also fix the level 0 SOUR record it points to so its _APID is renamed from NOTE and replace the REPO that went missing altogether....
        My initial post and recut was created in this forum to see if anyone was trying to use APID tags and I kept it brief, although it was quite long compared to other posts. IT did solve all the issues I was facing around sources and citations thereof. I didn't comment on REPO in my earlier post for the reason that it is a separate process and given the limited responses, I felt a secondary forum post of limited use.

        I have source entries with one APID tag but it appears from your reply, that you have many APID tags in one source, e.g.

        Source:
        0 @S56@ SOUR
        1 TITL California Birth Index, 1905-1995
        1 AUTH Ancestry.com
        1 PUBL Ancestry.com Operations Inc
        1 NOTE 1,5247::0
        1 NOTE Ancestry.com
        1 TEXT 1,5247::19893513
        2 CONT
        2 CONT 1,5247::19893513
        2 CONT
        2 CONT 1,5247::6985158
        <

        Comment


          #5
          Re: Reunion and Ancestry.com playing nice

          These are some of the before and after extracts. I had noted that Ancestry uses the citation detail to point to the relevant source in its database so the only aspects required for the source (or what works for me) is th


          Reunion exports the following

          Source citation (basically a citation detail of 1981 which is the same detail Ancestry would show for this record and the _APID tag)
          2 SOUR @S313@
          3 PAGE 1981_APID 1,1836::13364737

          Source (general source information with the much of the detail being saved as NOTEs. The REPOsitory is NOTE Ancestry and the APID is saved - I forgot that during my transition of data, I moved didn't save the ::0 entry, more on this later, hard to remember after 6 months of trial and error)
          0 @S313@ SOUR
          1 TYPE Electoral Roll
          1 TITL 1981 New Zealand Electoral Rolls about Nancye Dolores Cook
          1 NOTE Pakuranga, Auckland, New Zealand
          1 NOTE 1981
          1 NOTE Electoral Roll
          1 NOTE BAB microfilming. Microfiche publication, 4032 fiche
          1 NOTE http://search.ancestry.com/cgi-bin/s...91be062b6f54b8
          1 NOTE Original, Primary, Indirect
          1 NOTE Ancestry
          1 NOTE 1836::13364737
          1 TEXT Pakuranga, Auckland New Zealand, 1981, Electoral Roll, BAB microfilming. Microfiche publication, 4032 fiche, http://search.ancestry.com/cgi-bin/s...n=cook&uidh=he
          2 CONC d&rank=1&pcat=35&fh=2&h=13364737&recoff=6+7+8&qid= 6e99de04a63240f08b91be062b6f54b8
          1 CHAN
          2 DATE 3 NOV 2013

          After conversion, the Applescript simply splits the 1 PAGE line into 2 entries;
          Citation
          2 SOUR @S313@
          3 PAGE 1981
          3 _APID 1,1836::13364737

          Source (the only source data that ancestry needs as a minimum is the TYPE, TITLE, REPO (if you want it) and I save some detail in the TEXT tag). Note that the APID is not required, Ancestry website uses the citation to point to the source. The bare minimum is shown below but it is no more than Ancestry shows; I mimic it)
          0 @S313@ SOUR
          1 TYPE Electoral Roll
          1 TITL 1981 New Zealand Electoral Rolls about Nancye Dolores Cook
          1 MEMO Pakuranga, Auckland, New Zealand
          1 MEMO 1981
          1 MEMO Electoral Roll
          1 MEMO BAB microfilming. Microfiche publication, 4032 fiche
          1 MEMO http://search.ancestry.com/cgi-bin/s...91be062b6f54b8
          1 MEMO Original, Primary, Indirect
          1 REPO @R1@
          1 MEMO 1836::13364737
          1 NOTE Pakuranga, Auckland New Zealand, 1981, Electoral Roll, BAB microfilming. Microfiche publication, 4032 fiche, http://search.ancestry.com/cgi-bin/s...n=cook&uidh=he
          2 CONC d&rank=1&pcat=35&fh=2&h=13364737&recoff=6+7+8&qid= 6e99de04a63240f08b91be062b6f54b8
          1 CHAN
          2 DATE 3 NOV 2013

          The REPO data is again bare minimum. I tried saving more but it tended to ignore it, e.g. ADDR etc. I tried many different approaches but when I saved an Ancestry record against a person, I noted that a review of that saved record had minimum repository information. The Applescript changes the NOTE Ancestry into REPO @R1@ and obviously, you can have other repositories as well using the same method.

          0 @R1@ REPO
          1 NAME Ancestry.com

          I tried to save alot more data but in the end, I examined what Ancestry needed as a minimum such that the citation detail would appear as if I had simply saved the original Ancestry record against that person and not used Reunion to do so. It gives the illusion that the actual source information was all performed in Ancestry itself.

          This may not suit your needs but it is what I did, to try to get Reunion and Ancestry to work more aligned. It means that any Ancestry sourced record appears as it would had I saved it from their website, it shows the same citation detail (copy and paste back into Reunion) and the same minimum detail for repository.

          Is it perfect? no. The extra detail you could save via the web other than a save this record to a person hasn't been covered off. Also, Ancestry only allows one source to one person but the use of APID tags (same tag) against many citations means that a source can be linked to multiple people.

          PC

          Comment

          Working...
          X