Today I pushed some updates to:
These updates make it so that the Twitter, Foursquare and Instagram importers are now dynamically identifying and indexing People and Places, and marking them with a taxonomy within WordPress. I’ve also added a new system for “reprocessing” old posts which Keyring imported, so that you can go back and perform some function on those posts without having to import them again. You’ll find reprocessing tools under Tools > Import > Reprocess Keyring Data.
Reprocessing works by using the locally-stored copy of import data that is saved during the initial import of everything. The system is fully hookable, so you can add other reprocessing routines in via plugin. The core file comes bundled with one that attempts to address an old JSON-data-escaping issue, and I’ve added extensions to the importers listed above which allow you to go back and reprocess your posts for People/Places.
If you’re going to use them, I suggest you run the first one first, then you can run the others in any order you like. Doing the first one first will just make sure that as much of your data as possible is processable.
It’s worth mentioning that if you use these reprocessors, they can take a while (especially if you have a lot of data already), and that they will likely create a lot of new data (in the form of People and Place terms being created in their respective taxonomies). After running all of them over all of my data, I have almost 1,800 People and just over 3,000 Places in my database.
The other tool added in this upgrade is the ability to merge terms, which becomes important with all of this data.
When the importers are dynamically adding People and Places, they only match based on known identifiers. This means that you’re likely to end up with duplicate entries, especially if you’re processing multiple services (e.g. Foursquare and Instagram). Using the merge tool, you can browse through your entries and select 2 or more, then use the Bulk Actions drop-down to select “Merge” and hit “Apply”. Terms will be merged together as intelligently as possible, which basically means that the shortest slug of the group will be kept, and the longest strings for any conflicting fields will be kept. You can of course edit the resulting composite term afterwards and tweak things as you see fit. If you’re looking for a shortcut to identify duplicate entries, try searching for “-2”, which will give you a list of duplicates, then you’ll need to search for something that will bring up each of the dupes, select, merge, repeat. It’s a little bit tedious, but you’ll only need to do it once for each duplicate, and all future imports should match against the composite entry.
Oh, and one last thing — I threw in a quick map on the details page for Places, which provides a nice quick, visual confirmation that it’s the correct location. For now it’s using a very basic OpenStreetMap example, but I might switch it out to Leaflet at some point, which is pretty nice.
While I typically try to follow the Indieweb tenet of posting on my own site first and syndicating the content out to the social silos, I really like what you’re doing here with these plugins/code and have been meaning to play around with KeyRing. I’ve added these two plugins to the WordPress portion of the Indieweb wiki in hopes that others will see, use, and potentially contribute. Given your penchant to “own all the things”, you might like some of the tools that the Indieweb community is compiling/using if you hadn’t been aware of it previously. Some of the tools make it less onerous on individuals to support continual changes to social media APIs.
For the location related data, if you’re storing GPS in the recommended way, you might be interested in David Shanske’s Simple Location plugin which will take that data and allow you to display the related map (Google or Mapbox, others to come). His development repo is on Github. He made a major update recently and is continuing to iterate on it, but it has all the basics for use on a production site at this point.
Hi Chris, yep, I vaguely follow some of the Indieweb projects, and wrote a little bit about how I’m approaching it. I’ve know Tantek for years actually 😉 Thanks for adding these tools to the wiki!
That Simple Location plugin looks like a nice addition to this, since I am indeed using the recommended meta fields to store geo data. The custom theme that I wrote has built in support for automatically adding a map to the end of a post that contains the appropriate metadata, which you can see http://dentedreality.com.au/2017/01/07/brunchin-with-akires/
A note for anyone coming across this: looks like it currently does not handle unicode characters very well at all (converts them to `u00XX` in both label and slug). I guess I’ll have a look at fixing that before doing a “real” release of the plugins 🙂