DuckDuckGo, Bing, Mojeek, and other search engines are not returning full Reddit results any more.

  • Admiral Patrick@dubvee.org
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    3 months ago

    I just include the map-bot-user-agents.conf in my base nginx.conf so it’s available to all of my virtual hosts.

    When I want to enforce the bot blocking on one or more virtual host (some I want to leave open to bots, others I don’t), I just include a deny-disallowed.conf in the server block of those.

    deny-disallowed.conf
      # Deny disallowed user agents
      if ($ua_disallowed) { 
        return 444;
      }
    
    site.conf
    server {
      server_name example.com;
       ...
      include conf.d/includes/deny-disallowed.conf;
    
      location / {
        ...
      }
    }