Options to configure robots.txt on workhorse using a configmap

What does this MR do?

This adds the option to configure robots.txt file in workhorse - either appending or overwriting - using a volumeMount through a configMap.

Note this MR will require logic from another MR on the CNG workhorse image.

How this works: On installing charts/gitlab/charts/webservice the end-user can include append or to overwrite by supply configurations values under workhorse.robotsTXT.append or workhorse.robotsTXT.overwrite.

examples:

• to append:

workhorse:
  robotsTXT:
    append: |
       line 1
       line 2 

• to overwrite:

workhorse:
  robotsTXT:
    overwrite: |
       line 1
       line 2 

Related issues

#2626

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for omnibus-gitlab opened