Recent Social Importer Updates

I’ve been trying to make small improvements to the Keyring Social Importers package (and People & Places) that I maintain, and have made a number of them over the last few weeks. Here are some details of recent updates which you may have missed:

People & Places

  • Improved the labels being used for each taxonomy, so that you don’t get random mentions of “tags” in the WordPress UI.
  • Improved the add_place_to_post() method so that you can add multiple Places to a single Post.
  • Now exposing both the people and places taxonomies via the REST API.

Keyring Social Importers

  • Added a filter so that you can easily (and globally) disable downloading of full content for Instapaper articles.
  • Made it easy to inject custom CSS for a specific importer.
  • Added a Nest Camera service and importer. Including recent updates, it will download a snapshot from the specified camera(s) during the hour indicated, auto-tag it using the location of the camera, and also associate it with a Place if People & Places is co-installed.
  • The Instagram importer now handles video posts properly, and will download the full video and embed it into your posts. Bundled a reprocessor to fix old posts, which would have previously been handled as image posts.
  • Also made the Instagram importer link up People mentioned in captions (not just those who are properly tagged as being in a post).
  • Fixed a bug in the Twitter importer which was mangling newlines. Added a reprocessor to fix it in old posts as well.
  • Now exposing where a post was imported from in the REST API.
  • Added Places support to the TripIt importer, which associates each post with Places for each airport flown through on that trip.

Keyring Social Importers has been updated in the plugin directory (version 1.7, or get it from Github) and you can get the latest version of People & Places from Github (still not an “official” plugin yet).

You can see most of them in action on my site, Dented Reality, which uses them to aggregate most of my online social activity. The People & Places data is not directly exposed yet, but you can see it in the REST API output.

Screen Shot 2017-03-04 at 1.11.22 PM
Places support added to TripIt importer.
Screen Shot 2017-03-04 at 1.13.38 PM.png
Added a Nest Camera importer.
Screen Shot 2017-03-04 at 1.18.56 PM.png
Current list of data reprocessors.

SaaStr Annual, 2017

I attended the SaaStr Annual conference in San Francisco last week, and here are some of the notes I took in amongst the different sessions.

Continue reading “SaaStr Annual, 2017”

Peak Design Everyday Backpack 20L Review

Towards the end of last year, I jumped on the Kickstarter bandwagon for a new backpack. They blew their goal away, and it turned out to be a really professional release that went smoothly, and delivered on time. Major kudos to the folks at Peak Design.

I had previously been using a free backpack that was one of the swag items we gave away to some of our VIP clients, but wanted something a bit bigger, and more comfortable. The 20L version of the Everyday Backpack has thus far proven to be exactly that, and much more.

Minor Nitpicks

Let’s start with the very few things that I can fault this bag on. Here are the only ones I could come up with:

  • I find that the shoulder straps self-adjust/slide out of position on their own sometimes. So basically I’ll put the backpack on, and find that I need to tighten the shoulder straps (pull down on the ends). Not a big deal at all, but a minor annoyance on an otherwise pretty flawless bag.
  • Perhaps because of how I’m using the dividers, I find myself opening side pockets constantly to get things out. Again, not a big deal, and to be honest, I could probably reduce this by re-organizing the interior a bit, but the reality is that the side pockets are marginally awkward to get into once you put the bag down (e.g. you probably end up laying it flat on a desk or something to do it).
  • There is a small reflector strip at the bottom/front of the bag, but it doesn’t seem to reflect all that well, so I don’t know how visible it makes you as a cyclist, for example.
  • The side pockets are relatively small, so you can’t for example fit a full-sized Nalgene bottle in there, but this one from Miir fits pretty perfectly.

Things I Love

With that out of the way, let’s move on to all the things I really like about it!

  • I really like the size of the 20L. It’s just right to carry everything I could possibly want in a day, without giving me enough room to tote around a bunch of junk that I don’t need.
  • The materials all seem really high quality, especially the custom hardware (clips etc). The fabric is a weird, futuristic-feeling stuff, and the waterproofing seems to work pretty well (already got snowed/melted on once).
  • The zipper locking loops, while not useful all the time, are a pretty nifty addition, and it’s impressive how they serve dual-purpose with the external load-straps. Speaking of which…
  • The external load straps are a great idea, and surprisingly useful. I’ve already used them a few times for carrying a jacket/sweater after needing it in the morning, and then finding myself too hot in the afternoon.
  • I really love the shoulder straps, and their pivoting joint to the main bag. I find them perfectly comfortable, no matter how much weight I’ve put in the bag.
  • All the magnets are such a subtly-stylish way of handling closures, and make this thing feel like it’s from the future. Side pockets, base/front panel, interior pocket. Fwap.
  • At first I thought the top + side handles were overkill, but I’ve come to really appreciate them. They feel sturdy, and are easy to grab from any angle.
  • The luggage pass-through has already proven really handy, especially when combined with the side pockets (one of which becomes the “top” pocket, when you have the backpack mounted on the handle of a piece of rolling luggage. I travel a fair bit, so it’s good to be able to keep things organized when on the move.
  • The interior dividers are awesome! I thought they were kind of ridiculous when I was looking at the specs of the bag, and since I’m not a photographer I had assumed that I’d just pull them out and forget about them. Instead, I figured I’d give them a shot, and absolutely love them. They are the main form of organization for me within the bag, and I’ve tinkered around with flipping panels up/down, re-arranging them to create different sized compartments to suit different needs. Absolutely brilliant.

My Carry

  • I kept all the external load straps on there, although I’ve mainly used the bottom 2 so far.
  • I have the key fob carry still attached (with the little disc on my keychain), but only really use it when I travel (normally my keys live in my pocket).
  • I have the waist strap tucked into the side pockets, and don’t normally use it. Maybe if I find myself riding a bike a lot with the backpack (it’s winter, so that hasn’t happened yet), then I’ll use it more.
  • I use the chest strap pretty regularly, since I walk a lot with the pack on, and that helps distribute things, and make it more comfortable.
  • I’ll often carry a 13″ MacBook Pro, adaptor, 9.7″ iPad Pro in a keyboard case, a bunch of cables, and a few notebooks, plus a bunch of smaller things (plugs, memory cards, spare batteries, etc).

Overall, I think this is my favorite daily-carry/work backpack I’ve had thus far, and I’ve had a bunch. This is a great bag, and I’m looking forward to using it for years. Or at least until some other new fancy bag comes along 😉

Amazon Echo (Alexa) vs Google Home (Assistant)

I have both of these devices at home, and have had them each for a few months (at least now). I got the Amazon Echo first, so have had it for a lot longer. I grabbed a Google Home because they were on sale and I wanted to compare the 2 systems. Here’s an outline of my experience thus far.

Amazon Echo/Alexa

  • Far superior audio quality. The speaker blows the Google Home away.
  • Microphones/initial detection seem to be better, so you can yell from another room, from around a corner, etc, and Alexa will respond.
  • Earlier to the game with a developer program, so there are a lot of “skills” available, although a lot (most?) of them are pretty useless trivia type games.
  • Smart Home integration is much better so far. Includes an “internal” concept of devices, so you can create groups of devices natively within the Alexa app, and use them in commands, even if they’re different device types. For example I can group a LIFX light bulb, and a light that’s controlled through Wink, and have it all controlled as a single group via Alexa.
  • Wink hub integration is nice, and I centralize as much as possible through there.
  • Developing for Alexa is kind of cool. I built a few test skills using Lambda hosting, and was able to interact with Car2Go and pretty easily.
  • Radio integration (via TuneIn) is pretty nice, and being able to just “play triple j radio” (Australian station) is awesome.
  • I use Spotify for all my music, and it gets a little tedious having to say “on Spotify” for everything (because Alexa will default to Amazon Prime music otherwise).
  • It feels like total magic to walk into my house with music streaming from Spotify to headphones (via my phone), and then say “Alexa, play from Spotify” and it’ll just take over mid-track and keep playing.

Google Home/Assistant

  • Linking it up with a Chromecast allows for some really nice integrations.
  • No Wink integration yet leaves me pretty sad. It means that basically none of the home automation stuff that I have is accessible directly through the Home. I have to set things up as a shortcut in Wink, then as an IFTTT trigger against that shortcut. Pretty annoying.
  • I turned to IFTTT pretty quickly to do any of the interesting home automation stuff I wanted to play with, because of the lack of Wink integration. It works reasonably well, but is a little slower than I’d like since it has to get to Google, IFTTT, then (in my case), Wink, and finally control something in my house.
  • Integrations are only available for Nest (thermostat only), Philips Hue and SmartThings on the home automation front.
  • I like the lights/interaction with the Google Home a bit more. The colored dots/animations feel “friendlier”.
  • The touch-sensitive top of the Home is a nice addition. I have mine on my desk and find myself just tapping it quickly to pause, and sometimes using the spiral/swipe to adjust volume (although that’s a kind of awkward interaction).
  • I haven’t built anything on it yet, but Google’s looks really interesting, and provides a pretty nice interface (and learning system) compared to having to write/generate a ton of utterances with Alexa.
  • I don’t find the microphones/initial detection particularly good. I prefer to say “Hey Google” than “OK Google”, but either way I often find myself repeating it a few times to “get its attention”, and I also seem to see more “false-triggers” on the Google Home than I do on the Echo.
  • The app prompts me to download separate apps, which I guess provide some level of integration? I tried downloading the TuneIn one, but couldn’t play JJJ radio because it “could not be found”.
  • Setting Spotify as my default music source is nice, so now if I just ask for a track/artist, it always uses Spotify. I do find that often the first attempt to play music from Spotify doesn’t work though. Assistant will say “ok, playing from Spotify…” then just go quiet. If I repeat myself, it’ll work the next time. I have a feeling it’s related to the way Spotify handles multiple devices on the same account though, which seems to be a little bit funky with the Google Home in particular.
  • Google seems generally better at answering “random” questions, and seems to come up with something for a lot more queries than Alexa.
  • The set up process for the Home + Chromecast was a bit nicer than for the Echo.

h/t to Luca’s post that inspired me to write up my own experiences.

Social Importer Upgrade

Today I pushed some updates to:

  1. People & Places
  2. Keyring Social Importers

These updates make it so that the Twitter, Foursquare and Instagram importers are now dynamically identifying and indexing People and Places, and marking them with a taxonomy within WordPress. I’ve also added a new system for “reprocessing” old posts which Keyring imported, so that you can go back and perform some function on those posts without having to import them again. You’ll find reprocessing tools under Tools > Import > Reprocess Keyring Data.

Screen Shot 2017-01-08 at 9.50.45 PM.png

Reprocessing works by using the locally-stored copy of import data that is saved during the initial import of everything. The system is fully hookable, so you can add other reprocessing routines in via plugin. The core file comes bundled with one that attempts to address an old JSON-data-escaping issue, and I’ve added extensions to the importers listed above which allow you to go back and reprocess your posts for People/Places.

If you’re going to use them, I suggest you run the first one first, then you can run the others in any order you like. Doing the first one first will just make sure that as much of your data as possible is processable.

Screen Shot 2017-01-08 at 9.51.33 PM.png

It’s worth mentioning that if you use these reprocessors, they can take a while (especially if you have a lot of data already), and that they will likely create a lot of new data (in the form of People and Place terms being created in their respective taxonomies). After running all of them over all of my data, I have almost 1,800 People and just over 3,000 Places in my database.

The other tool added in this upgrade is the ability to merge terms, which becomes important with all of this data.

Screen Shot 2017-01-08 at 9.54.49 PM.png

When the importers are dynamically adding People and Places, they only match based on known identifiers. This means that you’re likely to end up with duplicate entries, especially if you’re processing multiple services (e.g. Foursquare and Instagram). Using the merge tool, you can browse through your entries and select 2 or more, then use the Bulk Actions drop-down to select “Merge” and hit “Apply”. Terms will be merged together as intelligently as possible, which basically means that the shortest slug of the group will be kept, and the longest strings for any conflicting fields will be kept. You can of course edit the resulting composite term afterwards and tweak things as you see fit. If you’re looking for a shortcut to identify duplicate entries, try searching for “-2”, which will give you a list of duplicates, then you’ll need to search for something that will bring up each of the dupes, select, merge, repeat. It’s a little bit tedious, but you’ll only need to do it once for each duplicate, and all future imports should match against the composite entry.

Oh, and one last thing — I threw in a quick map on the details page for Places, which provides a nice quick, visual confirmation that it’s the correct location. For now it’s using a very basic OpenStreetMap example, but I might switch it out to Leaflet at some point, which is pretty nice.

Screen Shot 2017-01-08 at 10.00.16 PM.png

People & Places

Over the years, I’ve been working on a system to aggregate data that I publish to other social networks/sites back into my control, on my own WordPress install. Thus far, that has resulted in the creation of Keyring (plugin) to provide an abstracted interface to all of the web services I’m interested in, Keyring Social Importers (plugin) to do the basics of importing the data from different places, and Homeroom (theme) to display it all. Today, I’ve been working on a system that will detect people who are mentioned in an interaction, and link them across posts using a custom taxonomy. It does the same for physical locations, so I’ve called it People & Places.

Essentially, this plugin is just a pair of custom taxonomies, with some specific ways of referring to things. Pretty basic. It gets more interesting though when you update Keyring Social Importers to the trunk version, which will now work in tandem with People & Places to link everything up. I wouldn’t recommend it on a production site just yet — there’s a lot of rough edges still.

When KSI is pulling in content from each service (currently looking at Twitter, Instagram and Foursquare), there’s a new block of code that makes sure People & Places is available, and then looks for certain pieces of data. If it finds them, it bundles up the details, and passes that along in the import process. When posts are actually inserted, it will attempt to link up that post to the People/Places it found. If the People already exist, then they’ll just be linked, in the same way tags work. If they don’t exist yet, then a new Person entry will be created, and that will be used.

I plan to add in a basic term-merging function, so that you can manually (maybe automatically?) identify “duplicate people” across different networks, and intelligently merge their entires (re-linking any posts involved), so that you build up a single, combined view of your interactions with a particular person. I envisage some interesting possibilities with the archive pages for these taxonomies, and that over time it will build a really interesting dataset of your interactions, the places you physically go, etc.

I’ll probably still move the code around a bit, and there are definitely some bugs around duplicates and handling things across different networks, but it seems to be working so far. This is also probably the time to figure out a decent way to allow re-processing of imported data from the raw copy that the importers save in postmeta. Installing this new code will start gathering data on new imported entries, but won’t go back and do the same on all the posts you’ve already got. Rather than deleting all that data and re-importing/processing everything, I’d like to have a simple way to re-process the raw data that’s already stored locally.

Snow-shoeing in Breckenridge, CO

Yesterday I went up to Breck with Erika. She went snowboarding, and I tried out my new MSR snowshoes. I started out at the Breck Nordic Center, and hiked for maybe 3 miles or so, before heading back to Peak 8 to meet up for bloody marys and fries. It was a gorgeous day, and snowshoeing was fun. I ran into a few people who said there were moose around, but I didn’t see them unfortunately. Being out there made me really curious about what it’d be like to do with a full pack on, and be going backpacking in full winter. That’s still on my list of things to try out. Here are some pics from the trails I took (Willow, Engleman, Peaks Trail).