By default, Drupal will have a public page for every taxonomy term that lists content tagged with it. This is great when the pages are styled and filled with content, but can be a little disappointing in other cases.
There is no permission you can disable for the page. It is implemented as a view, and you can configure it at /admin/structure/views/view/taxonomy_term and set access permissions there. You could also filter it to specific vocabularies or terms.