HTTP is the foundation of data communication for the World Wide Web, where hypertext documents include hyperlinks to other resources that the user can easily access. The Hypertext Transfer Protocol (HTTP) is an application protocol for the distribution of hypermedia. Hypermedia contrasts with multimedia, in that it is interactive and frequently contains hyperlinks to other hypermedia information systems.

Google utilizes this hypertext protocol in part of it's process to index and record the content existing on the World Wide Web.  Google's Googlebot crawler, for example, scans the robots.txt file when indexing websites to check for special instructions on what sections it should ignore. 

Google's exclusion protocol is quickly becoming the Internet standard. In July 2019 Google proposed making their Robots Exclusion Protocol (REP) the official standard by releasing their parsing application to the public and have now made their REP open source. Any URI based transfer protocol can use Google REP it's not limited to HTTP anymore and can be used for Constrained Application Protocol (CoAP) and FTP as well.

Frequently, these fail to give direct crawling instructions, creating a 'hole' in the Application-Layer Protocol Negotiation (ALPN) which can be exploited. Part of the ALPN process includes Transport Layer Security (TLS) (formally Secure Sockets Layer (SSL)), which are designed to provide communication security. The protocol is subject to abatement, delaying or excluding the target during the process.

Our TAP protocol is designed to communicate with the indexing robot and abate the transfer of targeted content, resulting in deferred association, causing the target association to deteriorate until it is deindexed.

A service of CleanSearch LLC https://cleansearch.com



CleanSearch Admin
CleanSearch® Customer Support
admin@cleansearch.com
800-544-7145 Office
561-288-0655 Cell
CleanSearch® LLC
2385 NW Executive Center Dr. Suite 100
Boca Raton, FL 33431