url-encoded proxy username

[tenjaa]

Hi!
we were exploring dynatrace with the java spring-music cf demo project and have some issues downloading the agent.

Our cloudfoundry instance requires an authenticating proxy to reach the internet.

So we set the HTTPS_PROXY env manually in the app manifest and during the staging process this is also picked up. But we guess that the buildpack has an issue with the @ that's urlencoded part of the username.
An example username would be testing@instanceidofproxyservice which we provide as HTTPS_PROXY: http://testing%40instanceidofproxyservice:password@proxyurl.com:8080.

In our proxy logs we see that the request to download the agent is using the username testing%40instanceidofproxyservice.
While curl is requesting with testing@instanceidofproxyservice, which works perfectly fine.

Logs:

Staging app and tracing logs...
   Downloading java_buildpack_cached...
   Downloaded java_buildpack_cached
   Cell d71bbeea-49a2-4931-a44b-20799ea6f2a1 creating container for instance ad9f93fd-d6f4-4f7e-b70b-d8401e40a295
   Cell d71bbeea-49a2-4931-a44b-20799ea6f2a1 successfully created container for instance ad9f93fd-d6f4-4f7e-b70b-d8401e40a295
   Downloading app package...
   Downloading build artifacts cache...
   Downloaded build artifacts cache (164B)
   Downloaded app package (49.9M)
   -----> Java Buildpack v4.26 (offline)
   -----> Downloading Jvmkill Agent 1.16.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/bionic/x86_64/jvmkill-1.16.0-RELEASE.so (found in cache)
   -----> Downloading Open Jdk JRE 11.0.5_10 from https://java-buildpack.cloudfoundry.org/openjdk/bionic/x86_64/openjdk-jre-11.0.5_10-bionic.tar.gz (found in cache)
          Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.0s)
          JVM DNS caching disabled in lieu of BOSH DNS caching
   -----> Downloading Open JDK Like Memory Calculator 3.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/bionic/x86_64/memory-calculator-3.13.0-RELEASE.tar.gz (found in cache)
          Loaded Classes: 26779, Threads: 250
   -----> Downloading Client Certificate Mapper 1.11.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.11.0-RELEASE.jar (found in cache)
   -----> Downloading Container Security Provider 1.16.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.16.0-RELEASE.jar (found in cache)
   [InternetAvailability]           WARN  Internet availability temporarily set to true: The Dynatrace One Agent download location is always accessible
   [DownloadCache]                  WARN  Unable to download https://xxx.live.dynatrace.com/api/v1/deployment/installer/agent/unix/paas/latest?include=java&bitness=64&Api-Token=xxx into cache /tmp/cache/final: 403 "Forbidden"
   [Buildpack]                      ERROR Finalize failed with exception #<RuntimeError: Unable to find cached file for https://xxx.live.dynatrace.com/api/v1/deployment/installer/agent/unix/paas/latest?include=java&bitness=64&Api-Token=xxx>
   Unable to find cached file for https://xxx.live.dynatrace.com/api/v1/deployment/installer/agent/unix/paas/latest?include=java&bitness=64&Api-Token=xxx
   Failed to compile droplet: Failed to run finalize script: exit status 1
   -----> Downloading Dynatrace One Agent latest from https://xxx.live.dynatrace.com/api/v1/deployment/installer/agent/unix/paas/latest?include=java&bitness=64&Api-Token=xxx Exit status 223
   Cell d71bbeea-49a2-4931-a44b-20799ea6f2a1 stopping instance ad9f93fd-d6f4-4f7e-b70b-d8401e40a295
Error staging application: App staging failed in the buildpack compile phase

[mmayr-at]

Hi @tenjaa,
I had a quick look at the code of the java build pack and most probably found the code causing the issue, see

java-buildpack/lib/java_buildpack/util/cache/download_cache.rb

Line 312 in 864477c

URI.parse(ENV['https_proxy'] || ENV['HTTPS_PROXY'] || '')

Here is the code in question, which does the proxy handling.

        def proxy(uri)
          proxy_uri = if no_proxy?(uri)
                        URI.parse('')
                      elsif secure?(uri)
                        URI.parse(ENV['https_proxy'] || ENV['HTTPS_PROXY'] || '')
                      else
                        URI.parse(ENV['http_proxy'] || ENV['HTTP_PROXY'] || '')
                      end

          @logger.debug { "Proxy: #{proxy_uri.host}, #{proxy_uri.port}, #{proxy_uri.user}, #{proxy_uri.password}" }
          Net::HTTP::Proxy(proxy_uri.host, proxy_uri.port, proxy_uri.user, proxy_uri.password)
        end

The URI.parse function does not decode the username/password. This can be easily proved here https://repl.it/repls/OpaqueAltruisticObservation. This is the reason why the encoded user/password is passed to the HTTP proxy. Unfortunately I'm not that into Ruby, so I don't know if this can be seen as a bug in the URI module or is intended behavior. Nevertheless I see two options to fix this:

1. Manually decode user/password before passing it to the Net::HTTP::Proxy method
2. Remove the special handling for the env variables, because Net::HTTP will automatically use them since Ruby 2.0.

I'd prefer option 2 here, since the buildpack already relies on Ruby 2.3.8? @nebhale can you please confirm? If yes, then I could adjust the code and create a PR.

[mmayr-at]

Hi @tenjaa,
I've created a PR, which should fix this issue by using Option 1 from above comment. Unfortunately the default handling of these environment variables in Ruby also does not decode the userinfo from the URL.

AFAIK you can try my changes with

cf push YOUR-APP -b https://github.com/REPOSITORY/BUILDPACK.git#BRANCH

which should be in this case

cf push YOUR-APP -b https://github.com/dynatrace-oss-contrib/java-buildpack.git#fix-url-encoded-proxy-auth

Please let me know about the results, thx!

[tenjaa]

Cool thanks for the fast help :)
Unfortunately our environement is quite locked down, so I am not able to use custom buildpacks :(
Hope to get this merged quickly!