I2P_Developers/i2p.www
3
0
Fork 5
Code Issues 27 Pull Requests Actions 1 Packages Projects Releases Wiki Activity

New proposal 160

Browse Source
This commit is contained in:
zzz
2022-01-03 07:26:45 -05:00
parent ab4868689d
commit 535f59d8c0
2 changed files with 385 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
i2p2www/pages/site/docs/applications/bittorrent.html
View File

@ -1,7 +1,7 @@
{% extends "global/layout.html" %}
{% block title %}{% trans %}Bittorrent over I2P{% endtrans %}{% endblock %}
{% block lastupdated %}{% trans %}January 2017{% endtrans %}{% endblock %}
{% block accuratefor %}0.9.28{% endblock %}
{% block lastupdated %}2022-01{% endblock %}
{% block accuratefor %}0.9.52{% endblock %}
{% block content %}
<p>{% trans -%}
@ -71,7 +71,9 @@ There are no known I2P clients or trackers that currently support UDP announce/r
<h2>{% trans %}Non-Compact Tracker Responses{% endtrans %}</h2>
<p>{% trans -%}
The non-compact response is just as in standard bittorrent, with an I2P "ip".
{%- endtrans %}</p>
{%- endtrans %}
This is a long base64-encoded "DNS string", probably with a ".i2p" suffix.
</p>
<p>{% trans -%}
Trackers generally include a fake port key, or use the port from the announce, for compatibility with older clients.
@ -296,32 +298,8 @@ are described below, and are subject to change.
Contact the I2P developers if you wish to develop a client or tracker supporting datagram announces.
{%- endtrans %}</p>
<p>{% trans -%}
A UDP tracker listens on two ports.
The "query port" is the advertised port, and is used to receive repliable (signed) datagrams, for the connect request only.
The "response port" is used to receive unsigned (raw) datagrams, and is the source port for all replies.
The response port is arbitrary.
A client sends and receives on a single port only.
It receives only unsigned (raw) datagrams.
Raw datagrams provides increased efficiency for replies since they contain tokens sent in the query, and need not be signed.
{%- endtrans %}</p>
<p>{% trans -%}
In the announce request, the 4-byte IP is replaced by a 32-byte hash, and the port is still present,
although it may be ignored by the tracker.
In the announce response, each 4-byte IP and 2-byte port is replaced by a 32-byte hash (compact peer info), and no port is present.
The client sends the announce request and scrape request to the source port in the announce response packet.
The connect request, connect response, scrape request, scrape response, and error response are the same as in BEP 15.
{%- endtrans %}</p>
<p>{% trans -%}
Source addresses in I2P cannot be spoofed, so it is possible to use a simplified protocol
with 2 packets instead of 4, omitting the connect request and response.
In this case, the announce request would be a repliable datagram sent to the tracker's query port,
and the tracker would not require a response port.
While this is more efficient, it would be more difficult to modify an existing tracker to support this mode.
The URL for the 4-packet-mode tracker would use standard "udp://" prefix.
The URL for a modified 2-packet-mode tracker would require a different prefix if both modes are supported in I2P.
<p>{% trans prop160=site_url('docs/spec/proposals/160) -%}
See <a href="{{ prop160 }}">Proposal 160</a>.
{%- endtrans %}</p>

378
i2p2www/spec/proposals/160-udp-trackers.rst Normal file
View File

@ -0,0 +1,378 @@
================================
UDP Trackers
================================
.. meta::
:author: zzz
:created: 2022-01-03
:thread: http://zzz.i2p/topics/1634
:lastupdated: 2022-01-03
:status: Open
:target: 0.9.54
.. contents::
Overview
========
This proposal is for implemention of UDP trackers in I2P.
Motivation
==========
As the user base in general and the number of bittorrent users specifically continues to grow,
we need to make trackers and announces more efficient so that trackers are not overwhelemed.
Bittorrent proposed UDP trackers in BEP 15 [BEP15]_ in 2008, and the vast majority
of trackers on clearnet are UDP-only.
A preliminary proposal for UDP trackers in I2P was posted on our bittorrent spec page [SPEC]_
in May 2014; this predated our formal proposal process, and it was never implemented.
This proposal simplifies the 2014 version.
It is difficult to calculate the bandwidth savings of datagrams vs. streaming protocol.
A repliable request is about the same size as a streaming SYN, but the payload
is about 500 bytes smaller because the HTTP GET has a huge 600 byte
URL parameter string.
The raw reply is much smaller than a streaming SYN ACK, providing significant reduction
for a tracker's outbound traffic.
Additionally, there should be implementation-specific memory reductions,
as datagrams require much less in-memory state than a streaming connection.
Design
============
This proposal uses both repliable and raw datagrams,
as defined in [DATAGRAMS]_.
BEP 15
-------
The message flow in [BEP15]_ is as follows:
.. raw:: html
{% highlight %}
Client Tracker
Connect Req. ------------->
<-------------- Connect Resp.
Announce Req. ------------->
<-------------- Announce Resp.
Announce Req. ------------->
<-------------- Announce Resp.
{% endhighlight %}
The connect phase is required to prevent IP address spoofing.
The tracker returns a connection ID that the client uses in subsequent announces.
This connection ID expires in one minute at the client, and in two minutes at the tracker.
This is not necessary in I2P because of repliable datagrams.
We propose two mechanisms for I2P, compatibility mode and fast mode.
Compatibility Mode
-------------------------
In compatibility mode, we keep the same message flow as BEP 15,
for ease of adoption in existing UDP-capable client code bases:
.. raw:: html
{% highlight %}
Client Tracker
Connect Req. -------------> (Repliable)
<-------------- Connect Resp. (Raw)
Announce Req. -------------> (Raw)
<-------------- Announce Resp. (Raw)
Announce Req. -------------> (Raw)
<-------------- Announce Resp. (Raw)
...
{% endhighlight %}
This mode is also useful if the client plans to send multiple announces
within one minute to a single tracker, as only the connect
message must be repliable.
I2P Fast Mode
-------------------------
In fast mode, we omit the connect phase, as it is not required to prevent address spoofing.
for ease of adoption in existing UDP-capable client code bases:
.. raw:: html
{% highlight %}
Client Tracker
Announce Req. -------------> (Repliable)
<-------------- Announce Resp. (Raw)
{% endhighlight %}
This mode omits a round-trip, but requires every announce request to be repliable.
Specification
=============
Repliable datagrams use I2CP protocol 17; raw datagrams use I2CP protocol 18.
Requests may be repliable or raw. Responses are always raw.
Connect Request
-----------------
Client to tracker.
16 bytes. Must be repliable. Same as in [BEP15]_.
.. raw:: html
{% highlight %}
Offset Size Name Value
0 64-bit integer protocol_id 0x41727101980 // magic constant
8 32-bit integer action 0 // connect
12 32-bit integer transaction_id
{% endhighlight %}
Connect Response
-----------------
Tracker to client.
16 bytes. Must be raw. Same as in [BEP15]_.
.. raw:: html
{% highlight %}
Offset Size Name Value
0 32-bit integer action 0 // connect
4 32-bit integer transaction_id
8 64-bit integer connection_id
{% endhighlight %}
The response MUST be sent to the I2CP "to port" that was received as the request "from port".
Announce Request
-----------------
Client to tracker.
98 bytes. Same as in [BEP15]_ except as noted below.
If preceded by a connect request/response, must be raw,
with the connection_id received in the connect response.
If NOT preceded by a connect request/response, must be repliable,
and the connection_id is ignored.
.. raw:: html
{% highlight %}
Offset Size Name Value
0 64-bit integer connection_id
8 32-bit integer action 1 // announce
12 32-bit integer transaction_id
16 20-byte string info_hash
36 20-byte string peer_id
56 64-bit integer downloaded
64 64-bit integer left
72 64-bit integer uploaded
80 32-bit integer event 0 // 0: none; 1: completed; 2: started; 3: stopped
84 32-bit integer IP address 0 // default
88 32-bit integer key
92 32-bit integer num_want -1 // default
96 16-bit integer port
{% endhighlight %}
Changes from [BEP15]_:
- connection_id is ignored if repliable
- IP address is ignored
- key is ignored
- port is probably ignored
The response MUST be sent to the I2CP "to port" that was received as the request "from port".
Do not use the port from the announce request.
Announce Response
-----------------
Tracker to client.
20+ bytes. Must be raw. Same as in [BEP15]_ except as noted below.
.. raw:: html
{% highlight %}
Offset Size Name Value
0 32-bit integer action 1 // announce
4 32-bit integer transaction_id
8 32-bit integer interval
12 32-bit integer leechers
16 32-bit integer seeders
20 + 32 * n 32-bit integer binary hash
{% endhighlight %}
Changes from [BEP15]_:
- Instead of 6-byte IPv4+port or 18-byte IPv6+port, we return
a multiple of 32-byte "compact responses" with the binary peer hashes.
The response MUST be sent to the I2CP "to port" that was received as the request "from port".
Do not use the port from the announce request.
I2P datagrams have a very large maximum size of about 16 KB;
however, for reliable delivery, datagrams larger than 4 KB should be avoided.
For bandwidth efficiency, trackers should probably limit the maximum peers
to about 50.
Scrape
----------
Scrape request/response from [BEP15]_ is not required by this proposal,
but may be implemented if desired, no changes required.
The scrape request is always repliable (unless there is a previous connect request/response)
and the scrape response is always raw.
Error Response
------------------
Error response from [BEP15]_ is not required by this proposal,
but may be implemented if desired, no changes required.
The error response is always raw.
Announce URL
------------
As in clearnet, UDP announce URLs are of the form "udp://host:port/path".
The path is ignored and may be empty.
If the ":port" part is omitted, use an I2CP port of 0.
Issues
=======
- Repliable datagrams do not support offline signatures.
That requires a separate proposal.
- This proposal does not support blinded destinations.
- This proposal offers two modes at the client's option.
An existing clearnet tracker such as "opentracker" would require more modifications
to support the fast mode. There is no way in the announce URL to indicate
support for only one mode.
- Compatibility mode may not be necessary, pending feedback from BiglyBT and
other developers. However, it would still save a lot of bandwidth
if it is used for several announces within a minute.
Repliable announces are about 450 bytes larger than raw announces.
Extensions
=============
Extension bits or a version field are not included.
Clients and trackers should not assume packets to be of a certain size.
This way, additional fields can be added without breaking compatibility.
Implementation guidelines
==========================
Clients
--------
For a given tracker hostname, a client should prefer UDP over HTTP URLs,
and should not announce to both.
Clients wihout existing BEP 15 support should implement
fast mode only, as it is much simpler.
Clients with existing BEP 15 support should require only small modifications.
Evaluate both fast and compatibility modes and choose
whatever is best for the existing code base.
If a client support DHT or other datagram protocols, it should probably
select a different port as the request "from port" so that the replies
come back to that port and are not mixed up with DHT messages.
Clients with a default list of opentrackers should update the list to
add UDP URLs after the known opentrackers are known to support UDP.
Clients may or may not implement retransmission of requests.
Retransmissions, if implemented, should use an initial timeout
of at least 15 seconds, and double the timeout for each retransmission
(exponential backoff).
Trackers
---------
Trackers must implement both compatibility mode and fast mode.
Trackers with existing BEP 15 support should require only small modifications.
For an integrated application (router and client in one process, for example the ZzzOT Java plugin),
it should be straightforward to implement and route the streaming and datagram traffic separately.
For an external tracker application that currently uses an HTTP server tunnel to receive
announce requests, the implementation could be quite difficult.
A specialized tunnel could be developed to translate datagrams to local HTTP requests/responses.
Or, a specialized tunnel that handles both HTTP requests and datagrams could be designed
that would forward the datagrams to the external process.
These design decisions will depend heavily on the specific router and tracker implementations,
and are outside the scope of this proposal.
Migration
=========
Existing clients do not support UDP announce URLs and ignore them.
Existing trackers do not support reception of repliable or raw datagrams, they will be dropped.
This proposal is completely optional. Neither clients nor trackers are required to implement it at any time.
Rollout
=======
The first implementations are expected to be in ZzzOT and i2psnark.
They will be used for testing and verification of this proposal.
Other implementations will follow as desired after the testing and verification are complete.
References
==========
.. [BEP15]
http://www.bittorrent.org/beps/bep_0015.html
.. [DATAGRAMS]
{{ spec_url('datagrams', True) }}
.. [SPEC]
{{ site_url('docs/applications/bittorrent', True) }}