Fun With Foam.
Instead of cutting up my insulation panels I figured it would be more useful to check the wall is fully sealed first. I suspected it probably wasn't given how fast rooms could cool down and, sure enough, I was right. The latest gap is between the floor of upstairs and the ceiling of downstairs, which explains why my thermal camera never spotted it. Today I pulled down a few last bits of drywall so I could get at as much of the wall as I could and figured the problem was most likely related to a porch support again. Sure enough, there's a gap in the wall. Not as bad as the first one I found - not even close - but enough that the air coming in through the gap was below 40 degrees.
So I prised the sheathing off the beam on the outside of the house and sprayed building foam as far into the gap as I could manage. Then I sprayed foam as far into the gap as I could manage from the inside. The remainder of the foam went behind the wall studs around the area to make sure that any remaining gaps wouldn't result in drafts blowing past everything. Then for good measure I sprayed some more into the general area, as best I could, from the upstairs room.
Within half an hour the coldest parts of the wall had measurably improved - the lowest temperature I could register increased from 37 degrees to 43 degrees, having steadily risen over the course of that half hour. I also noticed a couple of small cold spots that I'd missed before, that related to more gaps through the wall. So I chiseled the mortar out and replaced it with fresh.
Now I've got the heating in that part of the house turned off completely - it's pretty cold outside (last I saw it was in the 30s) and I'm interested to see if this latest draft is the last of the ugly ones. If it is I can think more about installing my insulation panels, which will be nice.