Can you see the iOS device actually hitting Cyrus with ANY request?

Yes I can see this.

HTTP/1.1 405 Method Not Allowed" (cnt-encoding=gzip; error=The requested method is not allowed for the URL.) [timing: cmd=0.000421 net=0.000027 total=0.000448]

I am getting the same issue through contacts app on Mac OS.

Is anybody else having this problem? I have tried reinstalling it but there is still the same issue.

What URLs is it trying to hit and with which methods? Do you have all of the prerequisites for Cyrus CalDAV and CardDAV installed? Have you enabled caldav and carddav in the http_modules option in imapd,conf? Have you enabled telemetry so we can see the entire HTTP request and response?

I have reinstalled it again to be safe and now it is connecting but not syncing. I think this must be an issue with apple clients.

Are you using calendars in the shared namespace maybe? If yes you may be hitting this bug:
#2373
Try applying the patch that dilyanpalauzov provided there. It really does wonders with DAV autodiscovery.

Please clarify if this is still relevant, after upgrading the client and server software.

Hello

I'm having the same issue, and I can't figure out how to solve it.

It is not related to shared account (I'm trying it with a "simple" personal account).

All is working fine with other clients than iOS or MacOS.
With those clients it seems they are not sending the Authorization header with credentials. And at the same time it seems in the logs ios_principals_url.txt in the message #2373 (comment) that the iPhone is actually authenticating.

Aug 24 10:35:17 nohostname https[31659]: [192.168.4.71] with "iOS/9.3.5 (13G36) dataaccessd/1.0"; "PROPFIND /dav/principals/user/ HTTP/1.1" (depth=0) => "HTTP/1.1 401 Unauthorized" (error=Must authenticate to access the specified target)
Aug 24 10:35:17 nohostname https[31659]: login: [192.168.4.71] karagian Basic+TLS User logged in SESSIONID=<nohostname.nodomain.com-31659-1535096117-1-15485954429730857089>

I saw that that there is a change since iOS 15 that the credentials are not sent without TLS w/ HTTP for security reason.

But when I try with the DAV client parameters (I saw this nextcloud issue):

• dav.server.co with user name user@otherdomain.co or user@server.co or user
• https://dav.server.co with user name user@otherdomain.co or user@server.co or user
• https://dav.server.co/dav/calendars/user/foo/Default/ with user name user@otherdomain.co or user@server.co or user

It ends wiith 405 on the url /principals

T 127.0.0.1:47976 -> 127.0.0.1:8080 [AP] #24
  PROPFIND /principals/ HTTP/1.1..Host: cyrus-back..Connection: close..Content-Length: 181..prefer: return=minimal..content-type: text/xml..depth: 0..brief: t..accept: */*..accept-encoding: gzip, deflate, br.
  .user-agent: iOS/15.6.1 (19G82) accountsd/1.0..accept-language: fr-FR,fr;q=0.9....<?xml version="1.0" encoding="UTF-8"?>.<A:propfind xmlns:A="DAV:">.  <A:prop>.    <A:current-user-principal/>.    <A:princip
  al-URL/>.    <A:resourcetype/>.  </A:prop>.</A:propfind>.                                                                                                                                                     
##
T 127.0.0.1:8080 -> 127.0.0.1:47976 [AP] #26
  HTTP/1.1 405 Method Not Allowed..Date: Mon, 31 Oct 2022 16:51:28 GMT..Connection: close, Upgrade..Upgrade: h2c..Vary: Accept-Encoding..Allow: OPTIONS, GET, HEAD..Content-Type: text/html; charset=utf-8..Cont
  ent-Encoding: br..Content-Length: 245.......<zipped content>              

Which may be normal because it is not querying on /dav/... url with a PROPFIND method.

So I have 2 questions :

• is it necessary to have the same domain name for user name as the dav server i.e. user@server.co with server dav.server.co?
• does anybody have an example of iOS client configuration that works with cyrus ? (it seems so in the issue Shared xDAV (CalDAV/CardDAV) resources are not discoverable #2373)

Thanks for your help.

does anybody have an example of iOS client configuration that works with cyrus ?

You can try the description at https://cal.aegee.org/h/en.html#apple : server aegee.org, username anonymous@aegee.org , any password. It works with iOS 14 and the only reason to provide a username is, that iOS insists on username. The server would return the same results without username (but different results with different username/password). I have not tried it with iOS 15. I could give you also valid username for read/write and you can try, if that works with iOS 15. My Cyrus system is however slightly patched (otherwise the anonymous login would not work).

thank you very much @dilyanpalauzov for the test.
So yes it works with iOS with your server (but without auth ?).

Still not working with our :(

We have an nginx proxy (other services are doing http) that we configured so there is no PROPFIND on /principals url :

• .well-known/* -> .well-known/*
• /dav/* -> /dav/*
• /* -> /dav/*

So now no more 405 but we have a 401. The phone is not authenticating and is not sending credentials in a basic auth Authorization header. It drives me nut.

So either we put the whole url and have a 401 :

########
T 127.0.0.1:36084 -> 127.0.0.1:8080 [AP] #84
  PROPFIND /dav/addressbooks/user/user@site.org/Default/ HTTP/1.1..Host: cyrus-back..Connection: close..Content-Length: 181..prefer: return=minimal..content-type: text/xml..depth: 0..brief: t..accept: */*..accept-encoding: gzip, deflate, br..user-agent: iOS/15.6.1 (19G82) accountsd/1.0..accept-language: fr-FR,fr;q=0.9....<?xml version="1.0" encoding="UTF-8"?>.<A:propfind xmlns:A="DAV:">.  <A:prop>.    <A:current-user-principal/>.    <A:principal-URL/>.    <A:resourcetype/>.  </A:prop>.</A:propfind>.                                                                                                                   
##
T 127.0.0.1:8080 -> 127.0.0.1:36084 [AP] #86  HTTP/1.1 401 Unauthorized..Date: Fri, 11 Nov 2022 17:50:41 GMT..Connection: close, Upgrade..Upgrade: h2c..Vary: Accept-Encoding..WWW-Authenticate: Basic realm="iroco.co"..Content-Type: text/html; charset=utf-8..Content-Length: 471....<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><title>401 Unauthorized</title></head><body><h1>Unauthorized</h1><p>Must authenticate to access the specified target</p><hr><address>Cyrus-HTTP/3.4.4 Cyrus-SASL/2.1.27 Lib/XML2.9.10 Jansson/2.13.1 Nghttp2/1.43.0 OpenSSL/1.1.1k Zlib/1.2.11 Brotli/1.0.9 Xapian/1.4.18 LibiCal/3.0 ICU4C/67.1 SQLite/3.34.1 Server at cyrus-back Port 8080</address></body></html>                                                                                                                       
####

or if we put the bare url :

####
T 127.0.0.1:36342 -> 127.0.0.1:8080 [AP] #4
  PROPFIND /.well-known/carddav HTTP/1.1..Host: cyrus-back..Connection: close..Content-Length: 181..prefer: return=minimal..content-type: text/xml..depth: 0..brief: t..accept: */*..accept-encoding: gzip, deflate, br..user-agent: iOS/15.6.1 (19G82) accountsd/1.0..accept-language: fr-FR,fr;q=0.9....<?xml version="1.0" encoding="UTF-8"?>.<A:propfind xmlns:A="DAV:">.  <A:prop>.    <A:current-user-principal/>.    <A:principal-URL/>.    <A:resourcetype/>.  </A:prop>.</A:propfind>.                                                                                                                                             
##
T 127.0.0.1:8080 -> 127.0.0.1:36342 [AP] #6
  HTTP/1.1 301 Moved Permanently..Date: Fri, 11 Nov 2022 18:30:53 GMT..Connection: close, Upgrade..Upgrade: h2c..Location: http://cyrus-back/dav/addressbooks..Vary: Accept-Encoding..Content-Length: 0....     
########
T 127.0.0.1:36344 -> 127.0.0.1:8080 [AP] #14
  PROPFIND /dav/ HTTP/1.1..Host: cyrus-back..Connection: close..Content-Length: 181..prefer: return=minimal..content-type: text/xml..depth: 0..brief: t..accept: */*..accept-encoding: gzip, deflate, br..user-a
  gent: iOS/15.6.1 (19G82) accountsd/1.0..accept-language: fr-FR,fr;q=0.9....<?xml version="1.0" encoding="UTF-8"?>.<A:propfind xmlns:A="DAV:">.  <A:prop>.    <A:current-user-principal/>.    <A:principal-URL/
  >.    <A:resourcetype/>.  </A:prop>.</A:propfind>.                                                                                                                                                            
##
T 127.0.0.1:8080 -> 127.0.0.1:36344 [AP] #16
  HTTP/1.1 207 Multi-Status..Date: Fri, 11 Nov 2022 18:30:53 GMT..Connection: close, Upgrade..Upgrade: h2c..Vary: Accept-Encoding, Brief, Prefer..Accept-Encoding: deflate, gzip..Preference-Applied: return=minimal..Content-Type: application/xml; charset=utf-8..Content-Encoding: br..Transfer-Encoding: chunked....<zipped content>                                                                                        
########
T 127.0.0.1:36346 -> 127.0.0.1:8080 [AP] #24
  PROPFIND /dav/principals/ HTTP/1.1..Host: cyrus-back..Connection: close..Content-Length: 181..prefer: return=minimal..content-type: text/xml..depth: 0..brief: t..accept: */*..accept-encoding: gzip, deflate,  br..user-agent: iOS/15.6.1 (19G82) accountsd/1.0..accept-language: fr-FR,fr;q=0.9....<?xml version="1.0" encoding="UTF-8"?>.<A:propfind xmlns:A="DAV:">.  <A:prop>.    <A:current-user-principal/>.    <A:principal-URL/>.    <A:resourcetype/>.  </A:prop>.</A:propfind>.                                                                                                                                                 
##
T 127.0.0.1:8080 -> 127.0.0.1:36346 [AP] #26
  HTTP/1.1 401 Unauthorized..Date: Fri, 11 Nov 2022 18:30:53 GMT..Connection: close, Upgrade..Upgrade: h2c..Vary: Accept-Encoding..WWW-Authenticate: Basic realm="iroco.co"..Content-Type: text/html; charset=utf-8..Content-Length: 471....<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><title>401 Unauthorized</title></head><body><h1>Unauthorized</h1><p>Must authenticate to access the specified target</p><hr><address>Cyrus-HTTP/3.4.4 Cyrus-SASL/2.1.27 Lib/XML2.9.10 Jansson/2.13.1 Nghttp2/1.43.0 OpenSSL/1.1.1k Zlib/1.2.11 Brotli/1.0.9 Xapian/1.4.18 LibiCal/3.0 ICU4C/67.1 SQLite/3.34.1 Server at cyrus-back Port 8080</address></body></html>         

Ending with 401 also, and we never see auth happening or Authorization headers sent.
Before we were using sabre server and it was working ok with iOS.
Maybe I will sniff how it was done there, and I'll put some more information here.

You can authenticate with secret abc and username aaa in domain aegee.org towards that server. In case this works for you, this is good for me.

After the last 401 is received, the client (ios) is supposed to send the username and password. If fact the client is supposed always and unconditionally the send the username and password, if the user enters those, but in practice username/password are send when 401 in encountered. I have no iOS 15.

Amendment: the server for authenticated iOS requests shall be set explicitly to webmail.aegee.org . It does return 401 more often than the server cal.aegee.org. cal.aegee.org always returns valid answers when no username/password is sent, 401 is returned only when username/password do not match. webmail.aegee.org returns 401 for any unauthenticated reques.

yes it works : reject demand with a bad password, and works fine with the credentials you indicated (you can remove them if you wish). Thanks again for this test. At least I know that it can work \o/

Did you patch the code for cyrus-http?

If you didn't I guess it comes from my nginx configuration, or imapd.conf

With sabre I see that iOS is authenticating after the first 401that indicate nonce auth :

Status: 401 Unauthorized
WWW-Authenticate: Digest realm="BaikalDAV",qop="auth",nonce="636f76cd6971f",opaque="d66d5f0524036afcb61420e358f990ce"

Then the next request is sending auth with nonce digest.

Authorization: Digest username="foo", realm="BaikalDAV", nonce="636f76cd6971f", uri="/dav.php", response="2a8297ba9769feae15b66c8a4764da7b", opaque="d66d5f0524036afcb61420e358f990ce", cnonce="ba8e5bee46bcb9ad6821ef30d6c3513c", nc=00000001, qop="auth

I also see WWW-Authenticate sent by cyrus but iOS is not sending back the credentials.

Status: 401 Unauthorized
WWW-Authenticate: Basic realm="iroco.co"

Do you have allowplaintext: yes in your imapd.conf ?

Did you patch the code for cyrus-http?

Yes, many times in the past and I cannot say now what change I made for what reason… So I am not going to provide here patches which might or might not be relevant to your case.

You can run Cyrus IMAP/httpd locally on your computer, without nginx/http proxy and see if things work, when connecting with iOS. I have done so in the past. You can also adjust your proxy to forward the requests to webmail.aegee.org . If that works, then you have configured the proxy server correctly and the problem is with httpd. If this does not work, then your proxy server configuration is suboptimal.

I will appreciate if you share information how iOS 14→15 changed, provided that you know this.

#2634 (comment) contains PROPFIND /principals/. This should have been PROPFIND /dav/principals/ and the prior requests shall have returned /dav/principal. In particular the DAV:current-user-principal request shall have returned that address. This is a hint on where to start investigating

Do you have allowplaintext: yes in your imapd.conf ?

allowplaintext: 0
If enabled, allows the use of cleartext passwords on the wire.

Yes I do have it, since the http reverse proxy does ensure the connection is TLS encrypted.

Yes I do have it, since the http reverse proxy does ensure the connection is TLS encrypted.

So we may have roughly the same configuration. I was thinking that it may be cyrus that would detect clear password with non TLS ciphered connection that could be the root cause. But if you do it the same way (TLS termination on the proxy) then this hypothesis is wrong.

I tried proxying your site with our reverse proxy, and it works fine. Thanks again for the test (and the idea). As the backend is TLS encrypted I could not "debug" what was done right with your server that might be wrong with mine.

I will appreciate if you share information how iOS 14→15 changed, provided that you know this.

Actually it was not a migration from iOS 14→15 but a migration from sabre/Baikal→cyrus that showed regressions for iOS 15 users. But I will share as much information as I can to help ppl with the same issue (I can maybe find some iOS 14 devices around to see whether the behaviour is similar on the PROPFIND discovery).

So my next steps will be to activate logs for the proxy pointing to your server, to check the headers/url. And if I cannot understand, then make it run on a local machine.

Hmm I think that I suspect HTTP/1.1 vs HTTP2 inconsistencies : when I return 401 or 301 with nginx proxy (listening in http2) then the negotiation is happening (it sends the Basic auth, and the backend is sending back a 207). But the mobile keeps not understanding the 207 answer. That means that each time the answer is coming from cyrus then iOS seems to not reading it. But when it comes from the proxy itself, it works fine.

For example if I do a curl like :

$ curl --http2 -k -iL -XPROPFIND -H 'Authorization: Basic <b64>' https://dav.iroco.co/dav/addressbooks/user/user@domain.io/Default/ -d '
<?xml version="1.0" encoding="UTF-8"?>
<A:propfind xmlns:A="DAV:">
  <A:prop>
      <A:current-user-principal/>
      <A:principal-URL/>
      <A:resourcetype/>
  </A:prop>
</A:propfind>'
curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)

If I put --http1.1 instead of http2 then I have the answer.

The communication between the mobile and proxy is in http2/TLS and the proxy is talking to cyrus in http 1.1.

Do you have a specific configuration with your proxy on that front?
Do you use a stream proxy?

I use nginx reverse proxy and you have connected to that proxy (proxies). Between nginx and httpd there is no encryption and therefore no http/2. Nginx does TLS and http/2 for the external communication. I do not think that I use streams.

Do you mean that once you disable http/2 in httpd and connect directly to it without proxy from iOS there is no problem?

Yay !! It works.
When the connection upgrade is hidden, iOS is adding the account.
Thank you very much @dilyanpalauzov for your support.

It was a proxy issue.
Cyrus is working fine with dav for iOS and MacOS.

So after refining the proxy configuration (no need actually for the WWW-Authenticate: Negotiate header) :

   client_max_body_size    10m;
   
   location /.well-known {
        proxy_pass http://cyrus-back/.well-known;                                                                                                                                                                
        proxy_redirect ~http://cyrus-back(.*) https://dav.iroco.co$1;
        proxy_hide_header Upgrade;
    }

    location /dav {
        proxy_pass http://cyrus-back/dav;
        proxy_hide_header Upgrade;
    }

No need also to translate / to /dav (for the 405) as iOS was trying urls because it was failing with http2.
Tested with iOS15 and iOS16, Mac OS X/10.14.6 (Mojave).

I guess this issue could be closed.