Ethernet frames larger than 64k
I have a several PCs in Gigabit Ethernet LAN, all are interconnected with single simple unmanaged L2 switch. No internet access. I want to build some realtime system, in which some PC should send a ~1MB messages to another, with such a property, that message is either delivered, either completely lost (in latter case I dont need to retransmit it, as it will be outdated).
More like "live feed from camera video+audio+some extra data from sensors". I want to use Ethernet protocol, as I dont need complex internet-like routing, and MAC addresses would be just suffient.
But Wiki said that max payload is only 46‑1500 octets. Can I overcome this limitation via custom-written software (e.g using netmap API), or is this a hardware-defined behaviour, that cannot be altered?
networking ethernet frame
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
|
show 1 more comment
I have a several PCs in Gigabit Ethernet LAN, all are interconnected with single simple unmanaged L2 switch. No internet access. I want to build some realtime system, in which some PC should send a ~1MB messages to another, with such a property, that message is either delivered, either completely lost (in latter case I dont need to retransmit it, as it will be outdated).
More like "live feed from camera video+audio+some extra data from sensors". I want to use Ethernet protocol, as I dont need complex internet-like routing, and MAC addresses would be just suffient.
But Wiki said that max payload is only 46‑1500 octets. Can I overcome this limitation via custom-written software (e.g using netmap API), or is this a hardware-defined behaviour, that cannot be altered?
networking ethernet frame
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
1
Unless an ethernet interface supports the non-standard jumbo frames, the interface will drop frames larger than 1514 bytes as malformed or a giant. Some devices support non-standard jumbo frames, but the maximum is usually 9000 bytes. The bad part is that different manufacturers do this differently, supporting different maximum sizes, sometime even on interfaces of the same device, e.g. some Cisco switches do not support, or only support 3000 bytes on 100 Mbps interfaces, but 9000 bytes on 1 Gbps interfaces on the same device. All interfaces in the path must support the size you use.
– Ron Maupin
Jan 7 at 19:22
@RonMaupin Thanks, I first wondered before trying to even starting implement. Can i somehow "flash" all adapters with new MTU or is this circuitry-defined? Where does this limitation "live"?
– xakepp35
Jan 7 at 19:38
This is a hardware limitation, and there is really nothing that can be done to change it. The IEEE ethernet standard is 1500 bytes, so any interfaces that work at larger sizes will be non-standard. You also cannot use jumbo frames across the public Internet. Another problem with trying to send large frames or packets is that you will lose some frames or packets, and then you must retransmit more data.
– Ron Maupin
Jan 7 at 19:42
@RonMaupin that is the topic, i either want a message delivered fully, either fully failed (based on checksum verification at recieve side). This theoretically must be done without any kind of feedback and retransmission. If i subdivide it into pieces that would be 3 times harder, because I will be forced to split 1MB across 700 packets of 1500 bytes. That will introduce 23800 octets of extra Ethernet headers, i may easily loss single packet, and ultimately I dont know if reordering could occur.
– xakepp35
Jan 7 at 19:47
That will not happen on a network. Frames are dropped all the time, and packets are lost all the time. The upper-layer protocols, e.g. TCP, or the application must consider that and take steps to determine when something is lost and resend it. There are also reasons, e.g. real-time protocols like VoIP or video, where you do not want to resend lost data, but for a file, you must detect and resend lost data.
– Ron Maupin
Jan 7 at 19:51
|
show 1 more comment
I have a several PCs in Gigabit Ethernet LAN, all are interconnected with single simple unmanaged L2 switch. No internet access. I want to build some realtime system, in which some PC should send a ~1MB messages to another, with such a property, that message is either delivered, either completely lost (in latter case I dont need to retransmit it, as it will be outdated).
More like "live feed from camera video+audio+some extra data from sensors". I want to use Ethernet protocol, as I dont need complex internet-like routing, and MAC addresses would be just suffient.
But Wiki said that max payload is only 46‑1500 octets. Can I overcome this limitation via custom-written software (e.g using netmap API), or is this a hardware-defined behaviour, that cannot be altered?
networking ethernet frame
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
I have a several PCs in Gigabit Ethernet LAN, all are interconnected with single simple unmanaged L2 switch. No internet access. I want to build some realtime system, in which some PC should send a ~1MB messages to another, with such a property, that message is either delivered, either completely lost (in latter case I dont need to retransmit it, as it will be outdated).
More like "live feed from camera video+audio+some extra data from sensors". I want to use Ethernet protocol, as I dont need complex internet-like routing, and MAC addresses would be just suffient.
But Wiki said that max payload is only 46‑1500 octets. Can I overcome this limitation via custom-written software (e.g using netmap API), or is this a hardware-defined behaviour, that cannot be altered?
networking ethernet frame
networking ethernet frame
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
asked Jan 7 at 19:17
xakepp35xakepp35
1491111
1491111
1
Unless an ethernet interface supports the non-standard jumbo frames, the interface will drop frames larger than 1514 bytes as malformed or a giant. Some devices support non-standard jumbo frames, but the maximum is usually 9000 bytes. The bad part is that different manufacturers do this differently, supporting different maximum sizes, sometime even on interfaces of the same device, e.g. some Cisco switches do not support, or only support 3000 bytes on 100 Mbps interfaces, but 9000 bytes on 1 Gbps interfaces on the same device. All interfaces in the path must support the size you use.
– Ron Maupin
Jan 7 at 19:22
@RonMaupin Thanks, I first wondered before trying to even starting implement. Can i somehow "flash" all adapters with new MTU or is this circuitry-defined? Where does this limitation "live"?
– xakepp35
Jan 7 at 19:38
This is a hardware limitation, and there is really nothing that can be done to change it. The IEEE ethernet standard is 1500 bytes, so any interfaces that work at larger sizes will be non-standard. You also cannot use jumbo frames across the public Internet. Another problem with trying to send large frames or packets is that you will lose some frames or packets, and then you must retransmit more data.
– Ron Maupin
Jan 7 at 19:42
@RonMaupin that is the topic, i either want a message delivered fully, either fully failed (based on checksum verification at recieve side). This theoretically must be done without any kind of feedback and retransmission. If i subdivide it into pieces that would be 3 times harder, because I will be forced to split 1MB across 700 packets of 1500 bytes. That will introduce 23800 octets of extra Ethernet headers, i may easily loss single packet, and ultimately I dont know if reordering could occur.
– xakepp35
Jan 7 at 19:47
That will not happen on a network. Frames are dropped all the time, and packets are lost all the time. The upper-layer protocols, e.g. TCP, or the application must consider that and take steps to determine when something is lost and resend it. There are also reasons, e.g. real-time protocols like VoIP or video, where you do not want to resend lost data, but for a file, you must detect and resend lost data.
– Ron Maupin
Jan 7 at 19:51
|
show 1 more comment
1
Unless an ethernet interface supports the non-standard jumbo frames, the interface will drop frames larger than 1514 bytes as malformed or a giant. Some devices support non-standard jumbo frames, but the maximum is usually 9000 bytes. The bad part is that different manufacturers do this differently, supporting different maximum sizes, sometime even on interfaces of the same device, e.g. some Cisco switches do not support, or only support 3000 bytes on 100 Mbps interfaces, but 9000 bytes on 1 Gbps interfaces on the same device. All interfaces in the path must support the size you use.
– Ron Maupin
Jan 7 at 19:22
@RonMaupin Thanks, I first wondered before trying to even starting implement. Can i somehow "flash" all adapters with new MTU or is this circuitry-defined? Where does this limitation "live"?
– xakepp35
Jan 7 at 19:38
This is a hardware limitation, and there is really nothing that can be done to change it. The IEEE ethernet standard is 1500 bytes, so any interfaces that work at larger sizes will be non-standard. You also cannot use jumbo frames across the public Internet. Another problem with trying to send large frames or packets is that you will lose some frames or packets, and then you must retransmit more data.
– Ron Maupin
Jan 7 at 19:42
@RonMaupin that is the topic, i either want a message delivered fully, either fully failed (based on checksum verification at recieve side). This theoretically must be done without any kind of feedback and retransmission. If i subdivide it into pieces that would be 3 times harder, because I will be forced to split 1MB across 700 packets of 1500 bytes. That will introduce 23800 octets of extra Ethernet headers, i may easily loss single packet, and ultimately I dont know if reordering could occur.
– xakepp35
Jan 7 at 19:47
That will not happen on a network. Frames are dropped all the time, and packets are lost all the time. The upper-layer protocols, e.g. TCP, or the application must consider that and take steps to determine when something is lost and resend it. There are also reasons, e.g. real-time protocols like VoIP or video, where you do not want to resend lost data, but for a file, you must detect and resend lost data.
– Ron Maupin
Jan 7 at 19:51
1
1
Unless an ethernet interface supports the non-standard jumbo frames, the interface will drop frames larger than 1514 bytes as malformed or a giant. Some devices support non-standard jumbo frames, but the maximum is usually 9000 bytes. The bad part is that different manufacturers do this differently, supporting different maximum sizes, sometime even on interfaces of the same device, e.g. some Cisco switches do not support, or only support 3000 bytes on 100 Mbps interfaces, but 9000 bytes on 1 Gbps interfaces on the same device. All interfaces in the path must support the size you use.
– Ron Maupin
Jan 7 at 19:22
Unless an ethernet interface supports the non-standard jumbo frames, the interface will drop frames larger than 1514 bytes as malformed or a giant. Some devices support non-standard jumbo frames, but the maximum is usually 9000 bytes. The bad part is that different manufacturers do this differently, supporting different maximum sizes, sometime even on interfaces of the same device, e.g. some Cisco switches do not support, or only support 3000 bytes on 100 Mbps interfaces, but 9000 bytes on 1 Gbps interfaces on the same device. All interfaces in the path must support the size you use.
– Ron Maupin
Jan 7 at 19:22
@RonMaupin Thanks, I first wondered before trying to even starting implement. Can i somehow "flash" all adapters with new MTU or is this circuitry-defined? Where does this limitation "live"?
– xakepp35
Jan 7 at 19:38
@RonMaupin Thanks, I first wondered before trying to even starting implement. Can i somehow "flash" all adapters with new MTU or is this circuitry-defined? Where does this limitation "live"?
– xakepp35
Jan 7 at 19:38
This is a hardware limitation, and there is really nothing that can be done to change it. The IEEE ethernet standard is 1500 bytes, so any interfaces that work at larger sizes will be non-standard. You also cannot use jumbo frames across the public Internet. Another problem with trying to send large frames or packets is that you will lose some frames or packets, and then you must retransmit more data.
– Ron Maupin
Jan 7 at 19:42
This is a hardware limitation, and there is really nothing that can be done to change it. The IEEE ethernet standard is 1500 bytes, so any interfaces that work at larger sizes will be non-standard. You also cannot use jumbo frames across the public Internet. Another problem with trying to send large frames or packets is that you will lose some frames or packets, and then you must retransmit more data.
– Ron Maupin
Jan 7 at 19:42
@RonMaupin that is the topic, i either want a message delivered fully, either fully failed (based on checksum verification at recieve side). This theoretically must be done without any kind of feedback and retransmission. If i subdivide it into pieces that would be 3 times harder, because I will be forced to split 1MB across 700 packets of 1500 bytes. That will introduce 23800 octets of extra Ethernet headers, i may easily loss single packet, and ultimately I dont know if reordering could occur.
– xakepp35
Jan 7 at 19:47
@RonMaupin that is the topic, i either want a message delivered fully, either fully failed (based on checksum verification at recieve side). This theoretically must be done without any kind of feedback and retransmission. If i subdivide it into pieces that would be 3 times harder, because I will be forced to split 1MB across 700 packets of 1500 bytes. That will introduce 23800 octets of extra Ethernet headers, i may easily loss single packet, and ultimately I dont know if reordering could occur.
– xakepp35
Jan 7 at 19:47
That will not happen on a network. Frames are dropped all the time, and packets are lost all the time. The upper-layer protocols, e.g. TCP, or the application must consider that and take steps to determine when something is lost and resend it. There are also reasons, e.g. real-time protocols like VoIP or video, where you do not want to resend lost data, but for a file, you must detect and resend lost data.
– Ron Maupin
Jan 7 at 19:51
That will not happen on a network. Frames are dropped all the time, and packets are lost all the time. The upper-layer protocols, e.g. TCP, or the application must consider that and take steps to determine when something is lost and resend it. There are also reasons, e.g. real-time protocols like VoIP or video, where you do not want to resend lost data, but for a file, you must detect and resend lost data.
– Ron Maupin
Jan 7 at 19:51
|
show 1 more comment
2 Answers
2
active
oldest
votes
1G (and up) ethernet provides Jumbo frame support with frame lengths up to 9000 bytes. I've seen some older Intel PCI cards (ancient stuff) which supported up to around 16k frames but I believe these are non-standard.
You WILL HAVE to split your messages into smaller chunks. Maximum frame length depends on HW (and no SW will work around this) but I don't think there is anything supporting 1M frames.
answered Jan 7 at 19:22
TomekTomek
22624
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
add a comment |
Ethernet (and for that matter, IP) was designed to have a fixed upper length on packet sizes so that no one can hog the shared medium for too long while transmitting some long arbitrary-length message.
Standard Ethernet frames can only have up to 1500 bytes of payload.
Standard IP datagrams can only be up to 64 KiBytes length. (But because there are so many Ethernet-like network links on the Internet, most of the time IP datagrams stay within the 1500 byte limit of Ethernet anyway, because fragmentation and reassembly of large IP datagrams is less efficient than just keeping datagrams within the data link layer's MTU.)
If your application deals in messages larger than that, you have to do message fragmentation/reassembly/validation yourself at your own layer. This is how it was designed and how everyone deals with it, and you'd be swimming upstream to try to hack your hardware to allow 1MB messages. Instead you should just realize that what you want is not something those lower layers provide for you, but they provide you the building blocks you need to build your own solution at your layer. Networking protocols and technologies are specifically designed in layers, with the lowest layers providing the bare minimum needed by that layer, so that upper layers aren't burdened by lower layers doing way more than the upper layers need. There's a seminal paper on this called "End-to-End Arguments in System Design" by Saltzer, Reed, and Clark. Anyone getting into networking at this level of detail should read it.
I'm sure there are plenty of message-framing libraries around to make it easy to pass arbitrary-length messages around via TCP or UDP, where the message-framing library takes care of fragmenting, reassembling, and verifying your messages.
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1391616%2fethernet-frames-larger-than-64k%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
1G (and up) ethernet provides Jumbo frame support with frame lengths up to 9000 bytes. I've seen some older Intel PCI cards (ancient stuff) which supported up to around 16k frames but I believe these are non-standard.
You WILL HAVE to split your messages into smaller chunks. Maximum frame length depends on HW (and no SW will work around this) but I don't think there is anything supporting 1M frames.
answered Jan 7 at 19:22
TomekTomek
22624
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
add a comment |
1G (and up) ethernet provides Jumbo frame support with frame lengths up to 9000 bytes. I've seen some older Intel PCI cards (ancient stuff) which supported up to around 16k frames but I believe these are non-standard.
You WILL HAVE to split your messages into smaller chunks. Maximum frame length depends on HW (and no SW will work around this) but I don't think there is anything supporting 1M frames.
answered Jan 7 at 19:22
TomekTomek
22624
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
add a comment |
1G (and up) ethernet provides Jumbo frame support with frame lengths up to 9000 bytes. I've seen some older Intel PCI cards (ancient stuff) which supported up to around 16k frames but I believe these are non-standard.
You WILL HAVE to split your messages into smaller chunks. Maximum frame length depends on HW (and no SW will work around this) but I don't think there is anything supporting 1M frames.
answered Jan 7 at 19:22
TomekTomek
22624
1G (and up) ethernet provides Jumbo frame support with frame lengths up to 9000 bytes. I've seen some older Intel PCI cards (ancient stuff) which supported up to around 16k frames but I believe these are non-standard.
You WILL HAVE to split your messages into smaller chunks. Maximum frame length depends on HW (and no SW will work around this) but I don't think there is anything supporting 1M frames.
answered Jan 7 at 19:22
TomekTomek
22624
answered Jan 7 at 19:22
TomekTomek
22624
answered Jan 7 at 19:22
TomekTomek
22624
answered Jan 7 at 19:22
TomekTomek
22624
22624
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
add a comment |
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
All jumbo frames are non-standard. The IEEE ethernet specification call for a maximum ethernet payload of 1500 bytes. Different manufacturers have adopted jumbo frames with no standard, and they implement it differently.
– Ron Maupin
Jan 7 at 19:24
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
eh.. unfortunately. I thought to use MAC broadcasting of 1MB frames, that would be great when multiple clients could recieve large realtime data simultaneously and efficiently. BTW, why 1500 and 9000? why not 65536!?
– xakepp35
Jan 7 at 19:27
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
That's likely very deep maths behind it, including wave effects of signal propagation in a copper cable and so on... Not for my head.
– Tomek
Jan 7 at 19:36
add a comment |
Ethernet (and for that matter, IP) was designed to have a fixed upper length on packet sizes so that no one can hog the shared medium for too long while transmitting some long arbitrary-length message.
Standard Ethernet frames can only have up to 1500 bytes of payload.
Standard IP datagrams can only be up to 64 KiBytes length. (But because there are so many Ethernet-like network links on the Internet, most of the time IP datagrams stay within the 1500 byte limit of Ethernet anyway, because fragmentation and reassembly of large IP datagrams is less efficient than just keeping datagrams within the data link layer's MTU.)
If your application deals in messages larger than that, you have to do message fragmentation/reassembly/validation yourself at your own layer. This is how it was designed and how everyone deals with it, and you'd be swimming upstream to try to hack your hardware to allow 1MB messages. Instead you should just realize that what you want is not something those lower layers provide for you, but they provide you the building blocks you need to build your own solution at your layer. Networking protocols and technologies are specifically designed in layers, with the lowest layers providing the bare minimum needed by that layer, so that upper layers aren't burdened by lower layers doing way more than the upper layers need. There's a seminal paper on this called "End-to-End Arguments in System Design" by Saltzer, Reed, and Clark. Anyone getting into networking at this level of detail should read it.
I'm sure there are plenty of message-framing libraries around to make it easy to pass arbitrary-length messages around via TCP or UDP, where the message-framing library takes care of fragmenting, reassembling, and verifying your messages.
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
add a comment |
Ethernet (and for that matter, IP) was designed to have a fixed upper length on packet sizes so that no one can hog the shared medium for too long while transmitting some long arbitrary-length message.
Standard Ethernet frames can only have up to 1500 bytes of payload.
Standard IP datagrams can only be up to 64 KiBytes length. (But because there are so many Ethernet-like network links on the Internet, most of the time IP datagrams stay within the 1500 byte limit of Ethernet anyway, because fragmentation and reassembly of large IP datagrams is less efficient than just keeping datagrams within the data link layer's MTU.)
If your application deals in messages larger than that, you have to do message fragmentation/reassembly/validation yourself at your own layer. This is how it was designed and how everyone deals with it, and you'd be swimming upstream to try to hack your hardware to allow 1MB messages. Instead you should just realize that what you want is not something those lower layers provide for you, but they provide you the building blocks you need to build your own solution at your layer. Networking protocols and technologies are specifically designed in layers, with the lowest layers providing the bare minimum needed by that layer, so that upper layers aren't burdened by lower layers doing way more than the upper layers need. There's a seminal paper on this called "End-to-End Arguments in System Design" by Saltzer, Reed, and Clark. Anyone getting into networking at this level of detail should read it.
I'm sure there are plenty of message-framing libraries around to make it easy to pass arbitrary-length messages around via TCP or UDP, where the message-framing library takes care of fragmenting, reassembling, and verifying your messages.
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
add a comment |
Ethernet (and for that matter, IP) was designed to have a fixed upper length on packet sizes so that no one can hog the shared medium for too long while transmitting some long arbitrary-length message.
Standard Ethernet frames can only have up to 1500 bytes of payload.
Standard IP datagrams can only be up to 64 KiBytes length. (But because there are so many Ethernet-like network links on the Internet, most of the time IP datagrams stay within the 1500 byte limit of Ethernet anyway, because fragmentation and reassembly of large IP datagrams is less efficient than just keeping datagrams within the data link layer's MTU.)
If your application deals in messages larger than that, you have to do message fragmentation/reassembly/validation yourself at your own layer. This is how it was designed and how everyone deals with it, and you'd be swimming upstream to try to hack your hardware to allow 1MB messages. Instead you should just realize that what you want is not something those lower layers provide for you, but they provide you the building blocks you need to build your own solution at your layer. Networking protocols and technologies are specifically designed in layers, with the lowest layers providing the bare minimum needed by that layer, so that upper layers aren't burdened by lower layers doing way more than the upper layers need. There's a seminal paper on this called "End-to-End Arguments in System Design" by Saltzer, Reed, and Clark. Anyone getting into networking at this level of detail should read it.
I'm sure there are plenty of message-framing libraries around to make it easy to pass arbitrary-length messages around via TCP or UDP, where the message-framing library takes care of fragmenting, reassembling, and verifying your messages.
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
Ethernet (and for that matter, IP) was designed to have a fixed upper length on packet sizes so that no one can hog the shared medium for too long while transmitting some long arbitrary-length message.
Standard Ethernet frames can only have up to 1500 bytes of payload.
Standard IP datagrams can only be up to 64 KiBytes length. (But because there are so many Ethernet-like network links on the Internet, most of the time IP datagrams stay within the 1500 byte limit of Ethernet anyway, because fragmentation and reassembly of large IP datagrams is less efficient than just keeping datagrams within the data link layer's MTU.)
If your application deals in messages larger than that, you have to do message fragmentation/reassembly/validation yourself at your own layer. This is how it was designed and how everyone deals with it, and you'd be swimming upstream to try to hack your hardware to allow 1MB messages. Instead you should just realize that what you want is not something those lower layers provide for you, but they provide you the building blocks you need to build your own solution at your layer. Networking protocols and technologies are specifically designed in layers, with the lowest layers providing the bare minimum needed by that layer, so that upper layers aren't burdened by lower layers doing way more than the upper layers need. There's a seminal paper on this called "End-to-End Arguments in System Design" by Saltzer, Reed, and Clark. Anyone getting into networking at this level of detail should read it.
I'm sure there are plenty of message-framing libraries around to make it easy to pass arbitrary-length messages around via TCP or UDP, where the message-framing library takes care of fragmenting, reassembling, and verifying your messages.
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
answered Jan 7 at 20:36
SpiffSpiff
76.7k10117162
76.7k10117162
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
add a comment |
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
What about IPv6 jumbogram, allowing up to 4GiB packets? tools.ietf.org/html/rfc2675 Its up to IP to divide packets, but what is the problem at lower Ethernet level, which should only care MAC-addressing and CRC functions, but not length!? Seems they overabused lower level Ethernet to do functions of IP protocol, havent read your book? (Ethernet is not nesesary IP)
– xakepp35
Jan 7 at 21:17
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
@xakepp35 It's completely appropriate for the link layer technology (Ethernet in this case) to impose a length limit, so that you can have a shared medium without unbounded latency/jitter from devices hogging the medium to transmit a giant message. In fact it was this breakthrough of limited-length packet switching that allowed the Internet to happen, and beat out the circuit-switched and message-switched networks that predated it.
– Spiff
Jan 7 at 21:36
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
I am not talking about internet, but about lacking flexibility. It would be great to rip off length limit from ethernet, because it presents in IP and it would allow to adapt (almost excellently fit) ethernet protocol and cheap hardware for local needs. On the internet there is no packets, other than IP. So why restricting it on lower level, while one simply may configure any router/gateway to reject/fragment larger frames?
– xakepp35
Jan 7 at 22:29
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
@xakepp35 Well at this point, anyone who needs arbitrary-length messages just uses a library that provides message-framing over TCP or UDP, which works fine on existing cheap commodity hardware, without needing new custom (read: expensive) hardware to support arbitrary-length messages. There's no market value to making the hardware change you're asking for, because people have already solved it at the application layer about 3 decades ago.
– Spiff
Jan 7 at 23:50
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1391616%2fethernet-frames-larger-than-64k%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Unless an ethernet interface supports the non-standard jumbo frames, the interface will drop frames larger than 1514 bytes as malformed or a giant. Some devices support non-standard jumbo frames, but the maximum is usually 9000 bytes. The bad part is that different manufacturers do this differently, supporting different maximum sizes, sometime even on interfaces of the same device, e.g. some Cisco switches do not support, or only support 3000 bytes on 100 Mbps interfaces, but 9000 bytes on 1 Gbps interfaces on the same device. All interfaces in the path must support the size you use.
– Ron Maupin
Jan 7 at 19:22
@RonMaupin Thanks, I first wondered before trying to even starting implement. Can i somehow "flash" all adapters with new MTU or is this circuitry-defined? Where does this limitation "live"?
– xakepp35
Jan 7 at 19:38
This is a hardware limitation, and there is really nothing that can be done to change it. The IEEE ethernet standard is 1500 bytes, so any interfaces that work at larger sizes will be non-standard. You also cannot use jumbo frames across the public Internet. Another problem with trying to send large frames or packets is that you will lose some frames or packets, and then you must retransmit more data.
– Ron Maupin
Jan 7 at 19:42
@RonMaupin that is the topic, i either want a message delivered fully, either fully failed (based on checksum verification at recieve side). This theoretically must be done without any kind of feedback and retransmission. If i subdivide it into pieces that would be 3 times harder, because I will be forced to split 1MB across 700 packets of 1500 bytes. That will introduce 23800 octets of extra Ethernet headers, i may easily loss single packet, and ultimately I dont know if reordering could occur.
– xakepp35
Jan 7 at 19:47
That will not happen on a network. Frames are dropped all the time, and packets are lost all the time. The upper-layer protocols, e.g. TCP, or the application must consider that and take steps to determine when something is lost and resend it. There are also reasons, e.g. real-time protocols like VoIP or video, where you do not want to resend lost data, but for a file, you must detect and resend lost data.
– Ron Maupin
Jan 7 at 19:51