I did not write about Web Development for quite a while. After reading an article about the use of X-Robot Tags today, I though it's time, again.
The Basics
You retrieve content from the Web by typing a Web address into your browser and the addressed Web server will send you the requested resource, like a regular HTML Web page, a PDF document, a JPEG image, a video or Flash Movie, a XML file, etc.
The browser and Web server communicate using he
HTTP (Hypertext Transfer Protocol ↑) and before the requested data is actually sent, they exchange HTTP Request and HTTP Response headers with information about the document, the browser, and the server. The
X-Robots-Tag Directive is an optional element (a directive) of such a HTTP Response Header. It was introduced by Google this year. Now, Yahoo has announced 2 weeks ago, that they support it, too.
You might recall that there is a
(X)HTML Meta Tag that allows to restrict the access control for search engine. But they only work for (X)HTML documents. Now the
X-Robots-Tag Directive allows the same for any non-(X)HTML resource like video-, audio-files, images, etc.