Monday, December 26, 2011

The agent learning loop

The great difficulty with software agents is the issue of how to program them to behave in an intelligent manner. Programming intelligence is difficult, tricky, and error-prone, when it is practical at all. Sometimes we are actually able to come up with simple, clever heuristics that seem to approximate intelligence (e.g., ELIZA), or maybe even heuristics for a very limited and constrained domain that come very close to approximating human-level intelligence (e.g., Deep Blue or Watson), but all too often our attempts at programming intelligence fall woefully short, are merely amusing, or are outright lame or horribly dysfunctional when situated in the real world. We will continue to pursue the vision of intelligent machines through programmed intelligence, but ultimately there is only one true path to true intelligence: the ability to learn.
Computer software that mimics intelligence focuses primarily on programming a library of encoded information and patterns that represent knowledge. That can enable a computer to answer questions or respond to environmental conditions, but only in a pre-programmed sense. The beauty of human-level intelligence is that the human mind has the ability to learn, to teach itself new facts, to recognize new patterns, to actually produce new knowledge.
We can also produce computers that embody quite a fair amount of the processing that occurs in the human mind, but we are still stymied by the vast ability of the mind to learn and produce knowledge and know-how itself.
Part of our lack of progress on the learning front is the simple fact that much of the demand for intelligent machines has been simply to replace humans for relatively mindless and rote activities. In other words, a focus on the economics of predictable production rather than creative and intuitive activities.
I would like to propose the overall sequence for a path forward towards intelligent machines. I call it the agent learning loop:
  1. We (humans) program an agent or collection of agents with some "basic" intelligence (knowledge and pattern recognition abilities.)
  2. We (humans) program those agents with the "basic" ability to "learn."
  3. We (humans) also program those agents with the ability to communicate their knowledge and learnings with other agents, as well as to simply be able to observe the activity of other agents and learn from their successes and failures.
  4. We (humans) program those agents with the ability to create new intelligent agents with comparable abilities of intelligence and abilities to learn. These agents are then able to learn and "know" incrementally more than us, their creators.
  5. These agents then create new agents, incrementally more intelligent than themselves, and the "loop" is repeated at step #1.
The theory is that each iteration of the loop incrementally increases the intelligence of the agents.
One key here is that multiple agents are needed at each step and that delegation, collaboration, and competition are critical factors in learning.
There is also a significant degree of Darwinian evolution in play here as well. True learning involves the taking of some degree of risk, such as with intuitive leaps, and sometimes even random selection when alternatives seem comparable in value, or even random selection on occasion when the choice might seem purely "rational." With a single agent risk is risky, but with multiple agents alternatives can be exploited in parallel. Agents that learn poorly or incorrectly will be at a disadvantage in the next generation and likely die off, although in some cases short-term "poor" behavior can sometimes flip over in future generations to have unexpected value as the environment itself evolves.
Communications between agents is critical, as is the ability to learn from failures. In fact, agents may "learn" to seek failure as a faster path to "successful" knowledge.

Wednesday, November 16, 2011

My remote control for the world

Many moons ago, years before smart phones, handheld computers, and even the Web, I had this conceptual idea for an imaginary device that I called my "remote control for the world." This was around 1985 or so. It wasn't intended to be a full-blown computer with a fancy GUI, but just a simple device to control just about everything you could imagine remotely. More of a scrolling list of devices and objects and a bunch of specific function buttons. Things like turning your home A/C or sprinklers on and off or checking the temperature in your home. Maybe even starting your car or checking its gas gauge. And your bank balance. Just about any setting or indicator that you have a personal interest in. I never bothered to write it up. I'm not sure why. But recently I was reminded of it (wanting to do something remotely) and I realized that even today its still isn't quite feasible with off-the-shelf technology.
The biggest breakthrough has been the Internet and Web, which provides a lot of the communications infrastructure that is needed. Wi-Fi and Bluetooth as well.
Small, hi-res graphical displays make the UI much more practical, but ultimately that was never really a major obstacle. A simple, hierarchical scrolling list and some tactile push buttons would work reasonably well.
We also have RFID for random objects, bad that is more passive and not interactive, yet.
And some household objects and "devices" are intelligent, but we are still in the "dark ages" for most devices and objects that we encounter in our daily lives. Did I lock my door? Sure, some "intelligent homes" can tell you that and control it remotely, but my vintage 1926 apartment door doesn't seem to have an Internet-ready interface.
Sure, you can now finally start your car remotely, but that is still more of a specialized app than a general feature of all devices and objects.
Thinking about my vision in today's world, I would like to "see" in my refrigerator and "see" in the relevant sections of the supermarket, or any store. Sure, we now have web cameras and other cheap cameras, but they still are not ubiquitous enough to make my vision a reality. And most cameras are still relatively low res and not even HD, lot alone photo-quality high-definition.
Yes, some cars now have rear cameras, but I'd like the same for when I'm simply walking down the street. Sure, you could "hack" that, but it's still not quite "there" in terms of a consumer-friendly handheld remote control device.
And I'd like to trivially (without some complicated and messy UI) be able to look around corners and even "through" buildings and walls. Sure, we have a lot of elements of the requisite technology, but we're still not even close to having enough of it off-the-shelf and readily packageable to satisfy my interests.
Right now, right this moment, I'd like to see where the sun is and check the temperature right outside my building. I can sort of do that with a bunch of browser clicks, but not quite and not as easily as with my original vision for my "remote control for the world." In other words, we can roughly approximate a subset of my vision, but not in any full-featured sense.
Maybe a rudimentary version of my remote control might be available in another five years, or ten years, but somehow all of our technological advances over the past 30 years have still not done a great job of integrating our physical environment and everyday devices and objects into one "seamless" network.

Friday, November 4, 2011

Moving on to reading about Ruby

Now that I've finished reading about Apache Hadoop and Apache Mahout, I've decided that I really need to know more than a little about Ruby and Ruby on Rails. I already do know a little, but I need to know a lot more about all the ins and outs and all the details of various features.
I figured I'll start with a basic tutorial just to get that out of the way. Something like Ruby in Twenty Minutes. This also involves downloading and installing Ruby itself on my Windows machine.
I started on the tutorial after downloading and installing Ruby, but realized that I would be much happier starting with the raw language specification since I am basically a compiler guy and I feel more comfortable knowing the strict specifications for white space, line terminators, comments, tokens, keywords, identifiers, literals, etc. before I start working with actual data and control structures. I found the Ruby Draft Specification.
I'll do a quick scan of the spec to get the lay of the land and then go back and go through the tutorial.
I may go back and carefully read the Ruby Wikipedia page first.
There are multiple tutorials, each emphasizing various aspects, including similarities to other programming languages. In my case, I'm actually interested in what language features of Ruby are unique or distinct from features of other languages.
I'd also like to identify one or more modest-sized actual open source Ruby applications so I can see first hand what good/great Ruby code looks like.
I'll focus on information that is available online as opposed to buying actually published books on Ruby,
Ultimately, I'd like to have a hard-core technical answer to the basic question of: What is Ruby good (best) for? I know people use Ruby and Ruby on Rails for user interfaces, but that's not a hard-core answer.

-- Jack Krupansky

Monday, October 31, 2011

Books, tutorials, and talks on Mahout

WatchMaker: Framework for genetic programming

Reading through the Apache Mahout wiki, I ran across the "Genetic Programming" section which listed WatchMaker, but the wiki page it pointed to had essentially zero descriptive information about WatchMaker itself or even a link to WatchMaker itself. Here's the link to the WatchMaker web page, which tells us that:
The Watchmaker Framework is an extensible, high-performance, object-oriented framework for implementing platform-independent evolutionary/genetic algorithms in Java. The framework provides type-safe evolution for arbitrary types via a non-invasive API. The Watchmaker Framework is Open Source software, free to download and use subject to the terms of the Apache Software Licence, Version 2.0.
Just to briefly summarize genetic programming (from the WatchMaker User Manual):
Evolutionary algorithms (EAs) are inspired by the biological model of evolution and natural selection first proposed by Charles Darwin in 1859. In the natural world, evolution helps species adapt to their environments. Environmental factors that influence the survival prospects of an organism include climate, availability of food and the dangers of predators.
Species change over the course of many generations. Mutations occur randomly. Some mutations will be advantageous, but many will be useless or detrimental. Progress comes from the feedback provided by non-random natural selection.
Evolutionary algorithms are based on a simplified model of this biological evolution. To solve a particular problem we create an environment in which potential solutions can evolve. The environment is shaped by the parameters of the problem and encourages the evolution of good solutions.
The field of Evolutionary Computation encompasses several types of evolutionary algorithm. These include Genetic Algorithms (GAs), Evolution Strategies, Genetic Programming (GP), Evolutionary Programming and Learning Classifier Systems.
The most common type of evolutionary algorithm is the generational genetic algorithm. We'll cover other EA variants in later chapters but, for now, all of the evolutionary algorithms that we meet will be some kind of generational GA.
The basic outline of a generational GA is as follows (most other EA variants are broadly similar). A population of candidate solutions is iteratively evolved over many generations. Mimicking the concept of natural selection in biology, the survival of candidates (or their offspring) from generation to generation in an EA is governed by a fitness function that evaluates each candidate according to how close it is to the desired outcome, and a selection strategy that favours the better solutions. Over time, the quality of the solutions in the population should improve. If the program is successful, we can terminate the evolution once it has found a solution that is good enough.
-- Jack Krupansky

Link to Apache Mahout Taste documentation

While reading through the Apache Mahout wiki I ran across a broken link to the Taste documentation on the Quickstart page. The Apache Mahout Taste documentation is here.
Apache Taste is a recommendation engine:
Taste is a flexible, fast collaborative filtering engine for Java. The engine takes users' preferences for items ("tastes") and returns estimated preferences for other items. For example, a site that sells books or CDs could easily use Taste to figure out, from past purchase data, which CDs a customer might be interested in listening to.
Taste provides a rich set of components from which you can construct a customized recommender system from a selection of algorithms. Taste is designed to be enterprise-ready; it's designed for performance, scalability and flexibility. Taste is not just for Java; it can be run as an external server which exposes recommendation logic to your application via web services and HTTP.

Leo Breiman's paper on Random Forests

I was reading the Apache Mahout wiki and got to the Breiman Example page and see that the link to Leo Breiman's paper (Random Forests) is broken. Here is the correct link to the paper. You can download the PDF as well.
The subject of the document is document classification.

Monday, October 24, 2011

Moving on from Hadoop to Mahout

I've finished reading the Apache Hadoop tuturial (from Yahoo). I didn't do any of the exercises, but at least I have more than a passing familiarity with what Hadoop is all about and how it is well-positioned to cope with Big Data.
Now, I'm moving on to reading up on Apache Mahout. Mahout's goal is  to build scalable machine learning libraries for recommendation mining, clustering, classification, frequent itemset mining, and similar purposes. There is actually a book on Mahout (Mahout in Action), but for now I'll focus on "mining" the Mahout wiki, which seems to have a lot of useful info which is likely sufficient for my immediate needs.
Mahout is implemented on top of Hadoop.
In the back of my head I'm thinking about entity extraction or named-entity extraction or named-entity recognition or NER as it is called. In theory, Mahout greatly facilitates NER.

Monday, October 17, 2011

Looking at Hadoop and Mahout

Since I finished my most recent contract work assignment on Friday I'm going to spend some time reading up on Hadoop and Mahout.
The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-avaiability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-availabile service on top of a cluster of computers, each of which may be prone to failures.
The Apache Mahout machine learning library's goal is to build scalable machine learning libraries.
Currently Mahout supports mainly four use cases: Recommendation mining takes users' behavior and from that tries to find items users might like. Clustering takes e.g. text documents and groups them into groups of topically related documents. Classification learns from exisiting categorized documents what documents of a specific category look like and is able to assign unlabelled documents to the (hopefully) correct category. Frequent itemset mining takes a set of item groups (terms in a query session, shopping cart content) and identifies, which individual items usually appear together.
In short, Big Data and cluster-based computing.
Today I'm reading through the Hadoop Tutorial.

-- Jack Krupansky

Thursday, July 14, 2011

The big problem with storage

As I continue to ponder the question of how to make real progress with software agent technology and a knowledge web, the big problem I keep coming back to is what I will call "The Big Problem with Storage", namely, how to achieve a degree of persistence in the digital networking domain comparable in robustness and reliability to storage in the physical world, and then to go a leap beyond that to achieve truly robust and reliable digital storage. Ultimately this includes communications reliability as well, but we can tolerate a little connectivity flakiness, but storage flakiness is not so tolerable since it generally cannot be recovered. What is needed is a fully redundant and diversified network storage scheme that is 100% robust and reliable so that people can have complete confidence that information and media stored on a digital network is even safer than the best storage in the real world.
I have written a proposal for a Vision for a Distributed Virtual Personal Data Storage, but it certainly doesn't appear as if even my limited proposal will happen any time soon.
I would add to this requirement that we are in desperate need of connectivity options that are far more reliable than the best offered today. Wired connectivity is probably the most reliable connectivity we have, but has diversity problems. Wireless has greater potential for diversity, but has coverage issues. The sad fact is that if you truly want "always-on" connectivity to your data you need to maintain a local copy on your local computer/network.

Sunday, June 26, 2011

Human Surrogate Travel

Surrogate travel is the concept of using a remotely-controlled robot to simulate travel and sensual experiences in a remote location. The user can move the robot around and listen and see what is around the robot. But this may be a significant logistical challenge given today's robotic and communications technology. So, why not use an actual human in place of the robot? The human robot would have one of more video cameras and microphones to provide sensual experiences to the user as well as a headset and microphone for communications with the user, so the user could audibly direct the human robot to move in a semi-mechanical or intelligent manner and the human robot could give the human user feedback as well.
This intermediate form of surrogate travel would be much more technologically feasible at the present time and in some cases maybe even more economical as well as more flexible. It might also be more socially acceptable than a free-roving robot.