Golang: how to check if an HTTP request is in the TCP payload bytes

https://stackoverflow.com/questions/39082404/golang-how-to-check-if-an-http-request-is-in-the-tcp-payload-bytes

I am using the gopacket package and every time I have a TCP packet I want to check if the payload contains an HTTP request. Is there an easy way to do that instead of writing my own parser? There is also a function (see: func ReadRequest(b *bufio.Reader)) which returns a Request struct but I do not know what kind of input I should use. tcp.Payload is the byte[] array that seems to have the information I need to parse (see the following example):

  1. // Get the TCP layer from this packet
  2. if tcpLayer := packet.Layer(layers.LayerTypeTCP); tcpLayer != nil {
  3. fmt.Printf("TCP ")
  4. // Get actual TCP data from this layer
  5. tcp, _ := tcpLayer.(*layers.TCP)
  6. srcPort = tcp.SrcPort
  7. dstPort = tcp.DstPort
  8. if tcp.SYN {
  9. fmt.Print("[SYN] ")
  10. }
  11. if tcp.ACK {
  12. fmt.Print("[ACK] ")
  13. }
  14. if tcp.FIN {
  15. fmt.Print("[FIN] ")
  16. }
  17. fmt.Printf("%s:%d > %s:%d ", srcIP, srcPort, dstIP, dstPort)
  18. fmt.Println(string(tcp.Payload))
  19. }

After sending an HTTP request I get the following output:

  1. PKT [001] TCP [SYN] 192.168.2.6:59095 > 192.168.3.5:80
  2. PKT [002] TCP [SYN] [ACK] 192.168.3.5:80 > 192.168.2.6:59095
  3. PKT [003] TCP [ACK] 192.168.2.6:59095 > 192.168.3.5:80
  4. PKT [004] TCP [ACK] 192.168.2.6:59095 > 192.168.3.5:80 GET /certificates/test.pdf HTTP/1.1
  5. User-Agent: Wget/1.15 (linux-gnu)
  6. Accept: */*
  7. Host: 192.168.3.5
  8. Connection: Keep-Alive

Any suggestions are welcome…


Do you mean you want to check if a packet has the start of an HTTP request? A single request is likely to span multiple packets. - JimB
Yes, let’s go for the simple case where the payload has only one HTTP request and without missing parts - tgogos
Yes, if the payload contains a the complete request you can feed it to http.ReadRequst. This simpler case isn’t really useful, since you’re operating on individual packets of a TCP stream. It fails immediately when you consider the possibility that a request can start in the middle of the packet payload, and it’s still very likely that the request will span multiple packets. HTTP is at a higher level than TCP, and you need to reassemble the stream (flow) before you can parse the HTTP correctly. - JimB


With help from this question: How to parse http headers in Go the following attempt with http.ReadRequest() works…

  1. if tcpLayer := packet.Layer(layers.LayerTypeTCP); tcpLayer != nil {
  2. tcp, _ := tcpLayer.(*layers.TCP)
  3. if len(tcp.Payload) != 0 {
  4. reader := bufio.NewReader(bytes.NewReader(tcp.Payload))
  5. httpReq, err := http.ReadRequest(reader)
  6. ...
  7. }
  8. }
ft_authoradmin  ft_create_time2019-06-19 11:23
 ft_update_time2019-06-19 11:23