on 15 October 20
Lately Iâ€™ve been pondering over search technologies a bit. How far do they really get us? With the amount of information that is pushed to us each day through various media, coupled with the fact that we additionally pull a lot of data to ourselves using search engines, TV, news, books and so on, we can easily feel like weâ€™re literally deluged with data. Everything seems interconnected, everything is responsive in some way, and yet there are still so many times that we are unable to find what we want.
So how far do common search technologies really get us and how much access do we really have to all the data thatâ€™s out there? The general dictionary definition of the word search would be on the lines of looking for something. But usually when we say run a search weâ€™re referring to a search within the online world, particularly using a Web search engine like Google, Yahoo, or Bing. There are also online searches that restrict themselves to their own domains, for example built-in searches offered within most portals, e-commerce sites, directories, and social networking sites. People who work with business applications, reports and databases may think in terms of database querying using SQL, or in terms of search boxes within an application.
It seems like the big internet search engines can throw up almost anything these days, but thatâ€™s not really the case at all. The fact is that they only retrieve data from the World Wide Web, and within that, itâ€™s only from the pages that they can index. This extent of the Web is called the surface Web. It has been estimated that the full extent of the Web may actually be 5000 times larger than that. Deep web sites remain beyond the reach of ordinary search engines for a number of reasons, but they can be accessed using networks like Tor.
Search technologies for common commercial use today are continuously being improved. With the advent of big data and an increase in the amount of public-domain data, there are new advances that enable searches to go beyond the usual limits of the world wide web and corporate boundaries, and into the public domain as well. Companies like Dragonglass have made this possible, and companies such as Ravn Systems and others are doing even more by using analytics and machine learning techniques to increase the contextual usefulness of search results.
But what I see as the biggest limitation is that all the searches we do are still largely either within the boundaries of the online world or completely in the offline (real) world. We can physically search for a person or thing in the real world, or we can look for references to that person or thing in the online world. To me, the ideal search technology would be one that seamlessly crosses the boundaries between the online and offline worlds.
As a simple example of such a search, who hasnâ€™t wondered at some point or the other about the exact current whereabouts of a dear one who might be driving on a long road trip through desolate areas by car at night? Sure, with cellphone technology and the assumption of signal availability the easiest thing would be to call and find out. But what if there was no cellphone coverage? The capability to do such searches are not available in the hands of common laypeople, but this is not completely because of the lack of technological advances but more likely because of restrictions commonly in place due to national security and personal privacy regulations and, of course, cost.
If the Hubble telescope can peer into the distant corners of the universe and process images to clear them of interstellar dust and debris might it not be possible to do even more right within the reaches of our own planet? In fact, I would dare to ask why we even have to use speech recognition to talk to a search engine. Why not just think of a search and have it done instantly, without moving a muscle?!
If that seems quite far fetched, just think about the possibilities we could have if a search was made using an integration of all the technologies that are in various states of evolution and maturity today:
- Thought transmission over a combination of near field and internet technologies: itâ€™s getting there slowly. This could be used to initiate searches without speaking or typing
- Searching across the online world is getting more sophisticated by the day thanks to cross-database searches and machine learning
- Deep web searches go into online libraries of books and many other types of resources that are unavailable on the indexed web
- Global satellite networks such as Iridium, Thuraya and Globalstar cover almost the whole planet
- Infrared cameras can see in the dark
- High resolution cameras can operate from extreme altitudes
- Technologies like Range-R can detect signs of life and movement even through walls
- Automatic target recognition systems (of the kind used in missiles) use a combination of cameras, lasers and sensors to home in on one object amongst many types of objects
- Image processing technologies incorporating object recognition can not only pick out classes of objects from a photo (or video) but identify them much more specifically as well. So can face recognition technologies.
- Primary radar and sonar can detect overhead or undersea objects and their movement even if they canâ€™t identify them
- Handwriting recognition, speech to text convertors, and character recognition readers make standardised text formats available
- GPS can pinpoint location very accurately
- Machine learning
And of course, we know that we have map views and street views already.
In short, we are able to search wide, search deep, and get up close. Itâ€™s easy to imagine, of course, why the use of many of these capabilities remain restricted in availability, but itâ€™s always fun to imagine what an ideal search infrastructure could do!
Share your perspective
Share your achievement or new finding or bring a new tech idea to life. Your IT community is waiting!