In Thursday’s column I mentioned some of the “hot” words that were trending in our www.nostracodeus.com word frequency analysis software.
One of the trends was related to “air travel” so we kept an eye out on the news sites for something to pop up out of the ordinary. Along came a report of a Boeing 737 emergency descent involving a Southwest Airlines jet and a curious report over here about a man reportedly falling out of a.Piper PA-46 Malibu 8-miles southeast of the Kendall-Tamiami executive airport. So that trend call seems to have been reasonably good.
Next was the word “hezbollah” which was trending and this showed up in the NY Times (and other media) as Hezbollah chief Hasan Nasrallah announced that his forces would not be withdrawing from Syria. Not that this announcement is even close to a surprise, but the language was trending off in that direction in advance. I’d give us a half-point on this; the headlines were there, but the Hezbollah make semi-regular headlines, so it’s interesting but in and of itself not totally convincing. Not as good as the “air travel” call.
The same thing can be said about the Taliban, to some extent. We see the headlines that the “New Leader of Pakistani Taliban could splinter group” alright. And yes, it was correct to anticipate some “news” from these folks, but again, in scoring how well one can see into the future, we continue tinkering with software and words.
A word of two about the process involved. The notion that we’re presently playing with is pretty simple: By simply counting word frequency across a large collection of news web sites, we believe that there may be hints of the future involved when words sort of (how to put this?) start popping out of the noise floor.
The basis for this is well-established, at least if you’ve studied Dean Radin’s works, and in particular if you’ve read “Time-Reversed Human Experience.”
There are many other good papers, however, on how humans perceive “time” and how events show up in time. One, for example, describes “The failure of Weber’s law in time perception and production.” (Bizo, Chu, Sanabria & Killeen, 2005).
If you don’t remember Weber’s Law (Weber-Fechner), Wikipedia puts it in a sentence:
Weber’s law states that the just-noticeable difference between two stimuli is proportional to the magnitude of the stimuli.
And this, you’d think would be a nice linear thing, but turns out it isn’t.
The reasons for looking into such nooks and crannies which range from quantum mechanics to behavioral psychology, to database design, and lands on one of a half-dozen servers where experimental databases are capturing “news” from dozens of sources, is that if you read what passes for “news” long enough, after a time you begin to notice a pattern of rhythm to it.
Where things get interesting is when you start marrying up some of the “new technology” that coming down the pike in the way of learning engines with large databases.
In a typical Nostracodeus collection process (which takes a couple of hours) the system may collection “reads” of several thousand pages of web content. This, next, is slammed into a fast database which then starts to sort out words in a number of ways.
The “direction” in the future we’re able to “look” is steered by the kinds of websites which are scanned. For example, in the predictions yesterday about the air travel, Hezbollah, and Taliban, the run (going from memory) had “eaten” something like 750 news sites.
What will be coming next will be a series of explorations which will explore more particular kinds of web sites. For example, we know there is a community under the heading “Science” and so Grady will be loading up a mission and running that to see what and how the language in the science community cycles.
Once we have a “cycle length” figured out, we can then anticipate when a particular topic might begin making headlines again, and then look for subtle changes in the occurrence of keywords immediately before events.
One area which may be undergoing a “just below the perception threshold” stage right now could be climate change. A few headlines have popped up recently, like the one : “IPPC chairman: We may “pass on a lousy, spoilt and defiled planet..”” Toss in the headline “Destruction of Brazil’s Amazon jumps 28%” and you can see how climate change is “lurking” just off stage, but ready to jump back into the headlines with the arrival of the next major disaster.
Take the recent typhoon which swept through the Philippines. A key story which didn’t quite make it into the “viral” or “top of mind” discussion among humans involved how the “Philippines delegation speech moves UN Climate Change Conference to tears…”
The future of such “news prediction” efforts seems bright. Web traffic to the Nostracodeus site continues to climb quickly and it’s likely due to people wanting a little more certainty or predictability about the future; beyond the odd hints which pop up in the mainstream media.
And there’s plenty of room for experimentation and development, too. While one approach may be based solely on talk among humans, this approach is genuinely different: It focuses on the mechanisms by which semi-ordinaria climbs the rungs of awareness – and seeks out who is helping(or retarding) along the way.
Once we get the “steering” down the way we like it (and related mathematical slicing and dicing of the database and an API built, the next step will be interfacing to “learning software” which can then take the analysis to the next step.
We are, you see, simply applying to the flow of news, what governments and major corporations are already doing in their development of Big Data handling of your individual website clicks. When you click on a page, say, about woodworking, and two sites later a pop-up comes up urging you to visit this woodworking site, or buy that fancy new tools, none of that is coincidental.
That’s the field in marketing that resides at the intersection of Big Data and Predictive Analytics trying to get you to stop sitting on your wallet.
Like it, or not, Predictive Analytics (PA) is here to stay.Why? Raw, naked power.
One of the PA books I’m reading at the moment (focusing on machine learning) recounts a college project where Supreme Court decisions by justice San Day O’Connor were fed into a “learning engine” which came up (without humans telling the machine how to do it) with an algorithm which was able to predict with 75% accuracy what her decisions would be in advance.
This compared to a panel of legal experts who were only able to guess with 59% accuracy, and then only after many tedious hours trying to comprehend what the underlying data sets (past court decisions) implied. I don’t know about you, but to me, that’s pretty damn impressive.
So that’s where Nostracodeus is heading at the moment – and maybe as early as tomorrow morning, Grady will start jotting down some of our development notes along the path.
Like most folks, I’m extremely interested in getting a better grip on the future in order to make better strategic living decisions of the sort outlined in the ebook I wrote with Gaye Levy titled 11 Steps to Living a Strategic Life.
It all comes down to this, I think: News, without context, projections, and well-considered actionable responses to its content – is simply a waste of time.
On the other hand, when you source not only news but actionable alternatives, well now you’re into this whole new area of “predictive living.” or PL – which we’ll get into in a future edition of Peoplenomics.com. Why there? Peoplenomics when comes right down to it, while focused on economic angles mainly, is about this whole “life management” stuff.
And like any other management problems, if you string enough good decisions together, you can end up in a winning position…which is what we aim for around here.
Friday at the WuJo:
This is a peach of a report – if you doin’t mind a little “throne room” aspect to oddities:
WuJU- Funny one here G man I had to laugh about it yesterday am. So I was taking a crap yesterday am in my master bedroom bathroom when I realized after the fact that there was literally one sheet of paper left on the roll. So I had to Waddle to the bathroom my son uses and finish the job in there. I had to go to the linen close after to get some new rolls to bring to my bathroom when I walk in my crap is gone and flushed with no toilet noise and there is a full roll on the rack. I just had to laugh and say out loud. Ok I am glad you care about my craps now! So either I am nuts which is entirely possible or the universe has a dandy sense of humor and is enjoying messing with me..
Hmmm…well, the Universe is flush with surprises…
Cheap Web Hosting Reminder
From EWMD’s old timer at the helm:
This is a reminder that our annual birthday sale begins today and will end tomorrow at midnight. Results are already great and we hope you will be able to take advantage of it. It would also be very helpful to us if you can let your friends and family know about it via social media and other means.
We had to make a couple of corrections to our original announcement sent out earlier this week. Here is the up to date information about our sale:Basic hosting: $4.60 per month for the life of the account.
Standard hosting: $5.60 per month for the life of the account.
Cloud VPS Servers: 10% off
This is our BEST deal to date. Please use the following coupons at sign up to take advantage of our Birthday sale prices. The codes will only be good on Nov 14th and 15th, 2013:
Basic Hosting: FIB7EMTC72
Standard Hosting: 4K8KP6TOR9
Cloud VPS Hosting: E8PJHWWAX0
Start over at www.emwd.com.
Around the Ranch: Achilles and Salvage Rights
Due to the confines of real work and a 90-minute nap (being dead tired) we only made limited progress on the carpentering experiment that is coming together at the north end of our house, which is turning a 20X10-foot deck into a new room.
I think the highlight of the day was stepping off the deck/room wrong (backwards) and tearing a muscle in my heel. This morning, I hate to admit, I’m limping around like an old man. So today will be devoted to “real work” until the ankle is up to doing real work again; I figure about Sunday.
The other highlight of living out here at the end of the string was when Brent, our local delivery fellow, found the key-fob that the large national delivery outfit uses to prevent thefts, keep the doors locked and control the engine had stopped working.
Since he’d turned off the engine, helping me get a package out, it took about 40-minutes for a mechanic to come out with a new fob. Turns out that a new battery (which we installed) wasn’t good enough. Apparently chip-level components can go bad in things.
When the problem was called in, I talked to the mechanic to give him simplified directions on how to get here and told him I had a cutting torch fired up and was claiming salvage rights to the truck if it wasn’t gone in an hour. He only thought I was kidding.
Elaine served up hot cocoa. and we had a fine conversation about how in the “old days” trucks were more dependable: You had a diesel engine, gear shift, and battery. There wasn’t much that could go wrong.
But on the new rigs, first you have to check that the red wire is connected, and that turns on a mystery box with a flashing green light on it, which in turn talks to the key-fob, which in turn allows the vehicle to start.
It was another one of those reminders that comes along every so often that what we call Progress may not be so progressive at all. In the old days, between Brent, Panama, and me, we could have gotten the truck fired up one way, or the other. And if it came to it, we could have pushed it down the hill with the tractor and it could have been double-clutched back to running.
Kind of difficult to do that with an automatic transmission and anti-theft devices which could have been wired around with a little daylight, but then there’s be someone in management likely upset with people in the outback taking matters into their own hands.
So, instead, we had cocoa and a good laugh about how what’s sold as “progress” works. Just to accomplish the same task (get the truck back to central dispatch) instead of just “gittin’ ;er done” we involved at least three other people: A supervisor, a mechanic, and somewhere in Asia I suspect, a heavy-duty key-fob maker” and a couple of railroaders, longshoremen, and computer engineers to dream up this cockamamie stuff
But to the mechanic’s credit, I will have to put away the cutting torch this morning.
Ya’ll come back Monday for more common sense from the woods and write when you break even.
George george@ure.net
