• None
    • Moderate
    • rhel-base-utils-core
    • ssg_core_services
    • None
    • False
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • If docs needed, set a value
    • None
    • 57,005

      Description of problem:

      • Enable oauth2 support in fetchmail for RHEL8

      Why does the customer need this?

      • Microsoft Office 365 is eliminating basic auth for mail service.
        This breaks workflows using RHEL MUAs.

      How would the customer like to achieve this? (List the functional requirements here)

      • Enable oauth2 support in RHEL8 mail client "fetchmail".

      For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented.

      • Fetchmail can connect to Exchange/Office 365 mail using oauth2

      Is there already an existing RFE upstream or in Red Hat Bugzilla?

      • not aware of one

      Does the customer have any specific timeline dependencies and which release would they like to target?

      • The ideal timeline is short, unfortunately. There are indications that Microsoft may push the deadline by a few months, but no commitment yet.

      Is the sales team involved in this request and do they have any additional input?

      • No

      List any affected packages or components.

      • Fetchmail

      Would the customer be able to assist in testing this functionality if implemented?

      • Yes

            [RHEL-6845] Enable oauth2 support in fetchmail

            If you are a Red Hat customer or partner, and this issue is having a negative impact on your business, please follow up with the support team through an already open support case, or open a new support case if not already done so, providing additional business justification so the issue can be reconsidered.

            RHEL Jira bot added a comment - If you are a Red Hat customer or partner, and this issue is having a negative impact on your business, please follow up with the support team through an already open support case, or open a new support case if not already done so, providing additional business justification so the issue can be reconsidered.

            pm-rhel added a comment -

            Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

            pm-rhel added a comment - Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

            somsky@uw.edu added a comment -

            As this is an update to fetchmail itself, I'm also going to look
            into the procedure to submitting it to the upstream project.

            somsky@uw.edu added a comment - As this is an update to fetchmail itself, I'm also going to look into the procedure to submitting it to the upstream project.

            somsky@uw.edu added a comment -

            Created attachment 1918030 [details]
            Patch to fetchmail7 allowing use of OAuth2 in daemon mode

            The following patch to the fetchmail 7 development branch (aka "next")
            allows fetchmail to use the mutt_oauth2.py OAuth2 helper script from
            the mutt mailreader project to obtain and refresh OAuth2 access tokens
            in daemon mode.

            It does so by adding a special sentinel value, "!PASSWDEVAL", to the
            "preconnect" fetchmailrc option which causes fetchmail to rerun and
            reload the command defined by "passwordeval" prior to each connection
            attempt to the email server.

            With this patch and the mutt_oauth2.py script, one can configure
            fetchmail to retrieve mail from Google GMail in daemon mode thusly:

            set daemon 300
            poll imap.gmail.com
            proto imap
            auth oauthbearer
            user username@gmail.com
            preconnect "!PASSWDEVAL"
            passwordeval "~/bin/mutt_oauth2.py ~/.gmail.oauth2"
            :

            – WRSomsky <somsky@uw.edu> 2022-10-14

            somsky@uw.edu added a comment - Created attachment 1918030 [details] Patch to fetchmail7 allowing use of OAuth2 in daemon mode The following patch to the fetchmail 7 development branch (aka "next") allows fetchmail to use the mutt_oauth2.py OAuth2 helper script from the mutt mailreader project to obtain and refresh OAuth2 access tokens in daemon mode. It does so by adding a special sentinel value, "!PASSWDEVAL", to the "preconnect" fetchmailrc option which causes fetchmail to rerun and reload the command defined by "passwordeval" prior to each connection attempt to the email server. With this patch and the mutt_oauth2.py script, one can configure fetchmail to retrieve mail from Google GMail in daemon mode thusly: set daemon 300 poll imap.gmail.com proto imap auth oauthbearer user username@gmail.com preconnect "!PASSWDEVAL" passwordeval "~/bin/mutt_oauth2.py ~/.gmail.oauth2" : – WRSomsky <somsky@uw.edu> 2022-10-14

            somsky@uw.edu added a comment -

            Okay, here's what I've been able to accomplish.

            I've created a small patch against the fetchmail v7 sources
            (from https://gitlab.com/fetchmail/fetchmail.git; branch: next)
            which allows fetchmail to use the mutt_oauth2.py OAuth2 helper
            script from the mutt mailreader project to obtain and refresh
            OAuth2 access tokens in daemon mode.

            It does so by adding a special sentinel value, "!PASSWDEVAL",
            to the "preconnect" fetchmailrc option which causes fetchmail
            to rerun and reload the command defined by "passwordeval"
            prior to each connection attempt to the email server.

            With this patch and the mutt_oauth2.py script, one can configure
            fetchmail to retrieve mail from Google GMail in daemon mode thusly:

            set daemon 300
            poll imap.gmail.com
            proto imap
            auth oauthbearer
            user username@gmail.com
            preconnect "!PASSWDEVAL"
            passwordeval "~/bin/mutt_oauth2.py ~/.gmail.oauth2"
            :

            I'm submitting this patch as attachment fetchmail.wrs001.patch.

            It seems to work successfully for me, but it needs more testing
            to make sure there aren't any "gotchas" I've overlooked,
            so please try it out.

            somsky@uw.edu added a comment - Okay, here's what I've been able to accomplish. I've created a small patch against the fetchmail v7 sources (from https://gitlab.com/fetchmail/fetchmail.git ; branch: next) which allows fetchmail to use the mutt_oauth2.py OAuth2 helper script from the mutt mailreader project to obtain and refresh OAuth2 access tokens in daemon mode. It does so by adding a special sentinel value, "!PASSWDEVAL", to the "preconnect" fetchmailrc option which causes fetchmail to rerun and reload the command defined by "passwordeval" prior to each connection attempt to the email server. With this patch and the mutt_oauth2.py script, one can configure fetchmail to retrieve mail from Google GMail in daemon mode thusly: set daemon 300 poll imap.gmail.com proto imap auth oauthbearer user username@gmail.com preconnect "!PASSWDEVAL" passwordeval "~/bin/mutt_oauth2.py ~/.gmail.oauth2" : I'm submitting this patch as attachment fetchmail.wrs001.patch. It seems to work successfully for me, but it needs more testing to make sure there aren't any "gotchas" I've overlooked, so please try it out.

            somsky@uw.edu added a comment -

            I'm trying fetchmail v7.devel from the git repository next branch.

            If you use mutt_oauth2.py, you can set "passwordeval" in your .fetchmailrc file
            to be "<path_to>/mutt_oauth2.py <path_to>/<tokenfile>" and it will automatically
            be invoked on fetchmail startup w/out needing a separate execution to update the access token.

            I'm now trying to get fetchmail to rerun the passwdeval() function
            at the beginning of each connection attempt (by hacking into the preconnect setting)
            but there are some subtleties that make simple voodoo hacking not work.
            That might be something for others to try.

            somsky@uw.edu added a comment - I'm trying fetchmail v7.devel from the git repository next branch. If you use mutt_oauth2.py, you can set "passwordeval" in your .fetchmailrc file to be "<path_to>/mutt_oauth2.py <path_to>/<tokenfile>" and it will automatically be invoked on fetchmail startup w/out needing a separate execution to update the access token. I'm now trying to get fetchmail to rerun the passwdeval() function at the beginning of each connection attempt (by hacking into the preconnect setting) but there are some subtleties that make simple voodoo hacking not work. That might be something for others to try.

            (In reply to goger from comment #13)
            > Hello,
            >
            > I'm very interesting by your rpm but I can't download it. I've an ssl error
            > with wget or any web browser.
            >
            > Can you fix the problem or explain me how to dowanload ?
            >
            > Thank you very much.
            >
            > Guillaume

            Hi Guillaume,

            The srpm is obsolete anyway. Upstream stopped releasing alpha tarballs and
            suggests using git 'next' branch to test the code.

            I've prepared Copr repository with fetchmail packages for RHEL8/9 (unofficial,
            unsupported) based on this branch that also contain oauth2 related contrib files
            and documentation, see:
            https://copr.fedorainfracloud.org/coprs/vcrhonek/fetchmail-7.x-epel/

            Vitezslav Crhonek added a comment - (In reply to goger from comment #13) > Hello, > > I'm very interesting by your rpm but I can't download it. I've an ssl error > with wget or any web browser. > > Can you fix the problem or explain me how to dowanload ? > > Thank you very much. > > Guillaume Hi Guillaume, The srpm is obsolete anyway. Upstream stopped releasing alpha tarballs and suggests using git 'next' branch to test the code. I've prepared Copr repository with fetchmail packages for RHEL8/9 (unofficial, unsupported) based on this branch that also contain oauth2 related contrib files and documentation, see: https://copr.fedorainfracloud.org/coprs/vcrhonek/fetchmail-7.x-epel/

            colin.jenkins@antalis.co.uk added a comment -

            Thanks again Jan and Greg,
            I have this working now.
            When I reviewed with our Azure admin I was using the secret id and not the secret value in client_secret
            A new secret id/value was created and when I tried to use the secret value I got a different error when getting a new access token.
            BUT, when I made the secret value empty it worked.

            So, just in case anyone has similar issues, here is how it has worked for me.
            1) Use mutt_oauth2.py with --authorize and option devicecode to get a refresh token and initial access token (does not matter if client_secret is specified or not)
            2) Use mutt_oauth2.py with no switch and client_secret set as empty to get new access token
            3) To prevent the need to use the gpg passphrase, decrypt the refresh token and save
            4) Use the fetchmail-oauth2.py with the decrypted refresh token specified as refresh_token_file and client_secret empty in the parameter file to get a new access token
            5) My crontab script now runs as below to get a new access token when needed before runnning fetchmail

            >> ${BINDIR}/fetchmail-oauth2.py -c ${CFGDIR}/oauth2Config.properties --auto_refresh
            >>
            >> fetchmail

            colin.jenkins@antalis.co.uk added a comment - Thanks again Jan and Greg, I have this working now. When I reviewed with our Azure admin I was using the secret id and not the secret value in client_secret A new secret id/value was created and when I tried to use the secret value I got a different error when getting a new access token. BUT, when I made the secret value empty it worked. So, just in case anyone has similar issues, here is how it has worked for me. 1) Use mutt_oauth2.py with --authorize and option devicecode to get a refresh token and initial access token (does not matter if client_secret is specified or not) 2) Use mutt_oauth2.py with no switch and client_secret set as empty to get new access token 3) To prevent the need to use the gpg passphrase, decrypt the refresh token and save 4) Use the fetchmail-oauth2.py with the decrypted refresh token specified as refresh_token_file and client_secret empty in the parameter file to get a new access token 5) My crontab script now runs as below to get a new access token when needed before runnning fetchmail >> ${BINDIR}/fetchmail-oauth2.py -c ${CFGDIR}/oauth2Config.properties --auto_refresh >> >> fetchmail

            jan@jankratochvil.net added a comment -

            For the refresh each 1 minute I am using:

            python3 ~/azul/fetchmail-git/contrib/fetchmail-oauth2.py -c /home/azul/.fetchmail-oauth2 --auto_refresh

            .fetchmailrc:
            auth oauthbearer username jkratochvil@azul.com passwordfile "/home/azul/.fetchmail-token"

            .fetchmail-oauth2:
            ...
            refresh_token_file=/home/azul/.fetchmail-refresh
            access_token_file=/home/azul/.fetchmail-token
            ...

            jan@jankratochvil.net added a comment - For the refresh each 1 minute I am using: python3 ~/azul/fetchmail-git/contrib/fetchmail-oauth2.py -c /home/azul/.fetchmail-oauth2 --auto_refresh .fetchmailrc: auth oauthbearer username jkratochvil@azul.com passwordfile "/home/azul/.fetchmail-token" .fetchmail-oauth2: ... refresh_token_file=/home/azul/.fetchmail-refresh access_token_file=/home/azul/.fetchmail-token ...

            colin.jenkins@antalis.co.uk added a comment -

            Thanks Greg,
            This is what I get when my access token expires and I try to get a new one.
            I am going to check the client secret value with my Azure admin shortly. But this works when I use "--authorize" and was copied and pasted originally, so should be no typo.


            ./mutt_oauth2.py cfg/o365-refresh-token-20220927
            401 Unauthorized
            invalid_client
            AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '
            Trace ID: 67856476-3302-49a8-a71c-c2f1a5304c00
            Correlation ID: 689175cf-f48b-459b-8eaa-b4ec86484de1
            Timestamp: 2022-09-27 11:50:51Z
            Perhaps refresh token invalid. Try running once with "--authorize"

            $ file cfg/o365-refresh-token-20220927
            cfg/o365-refresh-token-20220927: PGP RSA encrypted session key - keyid: B7A9EBCD 4466597E RSA (Encrypt or Sign) 2048b .

            colin.jenkins@antalis.co.uk added a comment - Thanks Greg, This is what I get when my access token expires and I try to get a new one. I am going to check the client secret value with my Azure admin shortly. But this works when I use "--authorize" and was copied and pasted originally, so should be no typo. – ./mutt_oauth2.py cfg/o365-refresh-token-20220927 401 Unauthorized invalid_client AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app ' Trace ID: 67856476-3302-49a8-a71c-c2f1a5304c00 Correlation ID: 689175cf-f48b-459b-8eaa-b4ec86484de1 Timestamp: 2022-09-27 11:50:51Z Perhaps refresh token invalid. Try running once with "--authorize" – $ file cfg/o365-refresh-token-20220927 cfg/o365-refresh-token-20220927: PGP RSA encrypted session key - keyid: B7A9EBCD 4466597E RSA (Encrypt or Sign) 2048b . –

              vcrhonek Vitezslav Crhonek
              rhn-support-duge Dushyant Uge
              Vitezslav Crhonek Vitezslav Crhonek
              RHEL SST CS base utils QE Bot RHEL SST CS base utils QE Bot
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: