Skip to content

Replace Timeout.timeout with TCPSocket.open(open_timeout:) when available #224

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

osyoyu
Copy link
Contributor

@osyoyu osyoyu commented Jul 16, 2025

Resolves #6.

This patch replaces the implementation of #open_timeout from Timeout.timeout from the builtin timeout in TCPSocket.open, which was introduced in Ruby 3.5 (https://bugs.ruby-lang.org/issues/21347).

The builtin timeout in TCPSocket.open is better in several ways. First, it does not rely on a separate Ruby Thread for monitoring Timeout (which is what the timeout library internally does). Also, it is compatible with Ractors, since it does not rely on Mutexes (which is also what the timeout library does).

This change allows the following code to work.

require 'net/http'
Ractor.new {
  uri = URI('http://example.com/')
  http = Net::HTTP.new(uri.host, uri.port)
  http.open_timeout = 1
  http.get(uri.path)
}.value

In Ruby <3.5 environments where TCPSocket.open does not have the open_timeout option, I have kept the behavior unchanged. net/http will use Timeout.timeout { TCPSocket.open }.

Changes in behavior

On timeout, the raised Net::OpenTimeout's message has slightly changed, and also carrys a Errno::ETIMEDOUT as its cause.

/home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1670:in 'Net::HTTP#connect': Failed to open TCP connection to example.com:80 (Connection timed out - user specified timeout) (Net::OpenTimeout)
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1636:in 'Net::HTTP#do_start'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1625:in 'Net::HTTP#start'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:2378:in 'Net::HTTP#request'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1993:in 'Net::HTTP#get'
	from nethttptest.rb:13:in '<main>'
/home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1660:in 'TCPSocket#initialize': Connection timed out - user specified timeout (Errno::ETIMEDOUT)
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1660:in 'IO.open'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1660:in 'Net::HTTP#connect'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1636:in 'Net::HTTP#do_start'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1625:in 'Net::HTTP#start'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:2378:in 'Net::HTTP#request'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1993:in 'Net::HTTP#get'
	from nethttptest.rb:13:in '<main>'

Previously, it looked like this.

/home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1665:in 'TCPSocket#initialize': Failed to open TCP connection to example.com:80 (execution expired) (Net::OpenTimeout)
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1665:in 'IO.open'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1665:in 'block in Net::HTTP#connect'
	from /home/osyoyu/.rbenv/versions/3.4.5/lib/ruby/3.4.0/timeout.rb:185:in 'block in Timeout.timeout'
	from /home/osyoyu/.rbenv/versions/3.4.5/lib/ruby/3.4.0/timeout.rb:192:in 'Timeout.timeout'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1664:in 'Net::HTTP#connect'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1636:in 'Net::HTTP#do_start'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1625:in 'Net::HTTP#start'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:2378:in 'Net::HTTP#request'
	from /home/osyoyu/Development/rubyorg/net-http/lib/net/http.rb:1993:in 'Net::HTTP#get'
	from nethttptest.rb:13:in '<main>'

…able

This patch replaces the implementation of #open_timeout from Timeout.timeout from the builtin timeout in TCPSocket.open, which was introduced in Ruby 3.5 (https://bugs.ruby-lang.org/issues/21347).

The builtin timeout in TCPSocket.open is better in several ways than Timeout.timeout. It does not rely on a separate Ruby Thread for monitoring Timeout (which is what the timeout library internally does).

Furthermore, it is compatible with Ractors, as opposed to Timeout.timeout (it internally uses Thread::Mutex which can not be used in non-main Ractors).
This change allows the following code to work.

    require 'net/http'
    Ractor.new {
      uri = URI('http://example.com/')
      http = Net::HTTP.new(uri.host, uri.port)
      http.open_timeout = 1
      http.get(uri.path)
    }.value

In Ruby <3.5 environments where `TCPSocket.open` does not have the `open_timeout` option, I have kept the behavior unchanged. net/http will use `Timeout.timeout { TCPSocket.open }`.
@osyoyu
Copy link
Contributor Author

osyoyu commented Jul 16, 2025

This version needs ruby/ruby#13909. It's now merged!

@osyoyu osyoyu force-pushed the tcpsocket-open-timeout branch from 4666f41 to 728eb8f Compare July 19, 2025 00:22
s = begin
# Use built-in timeout in TCPSocket.open if available
TCPSocket.open(conn_addr, conn_port, @local_host, @local_port, open_timeout: @open_timeout)
rescue ArgumentError => e
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't come up with a better way to detect the absence of open_timeout: option. Is this a acceptable solution?

TCPSocket.open is basically (**args) from the perspective of Ruby, so Method#parameters wasn't an option:

irb(main):001> require 'socket'
=> true
irb(main):002> TCPSocket.method(:open).parameters
=> [[:rest]]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine, given this is only to keep things working for very old rubies, right, and the error message is not going to change for those rubies anyways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it's not only very old Rubies, but even 3.4 raises on a TCPSocket.open call with open_timeout. It is true that the situation is different in Ruby 2.x, where keyword arguments were not a argument of its own kind (workaround in 09bf573).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I thought your question was specifically for the Ruby 2.x workaround, since you need to parse a very generic error message, but I see how the detection for Ruby 3.4 is also brittle since it also involves parsing the error message, even if more specific. Unfortunately, I don't know of a better way, but I'd say this way is acceptable.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering that every single new HTTP request is going to go through this raise/rescue flow on ruby < 3.5, and that exceptions are kind of expensive, personally I think a better option is just to have test for RUBY_VERSION.to_f < 3.5 directly to see if we use the old Timeout.timeout. I get that testing version numbers directly is a bit distasteful, but when there's actual performance issues and brittleness issues on the line... I'd say its merited

@osyoyu osyoyu marked this pull request as ready for review July 19, 2025 00:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replacing Timeout.timeout in Net::HTTP#connect with socket timeout options
3 participants