Googlebot user-agent configuration

Project Description

Category
AWS

Googlebot user-agent configuration – Project Summary

This configuration project was to block or allow Google crawlers from accessing some content of the website pages by specifying Googlebot as the user-agent

Technical Components

Googlebot, Linux
Website security
Contact Us