Getting Started with the HoneyDB Payload History API (Part 2)
A deeper dive on HoneyDB's Payload History API Endpoints
In part one of this series, we began exploring the HoneyDB Payload History API, focusing on how to retrieve payload attributes based on year and month, and then retrieving the complete history of an attribute with its hash value. Building upon that knowledge, part two will cover the next set of endpoints, which filter attributes by service names and attributes names.
Services
The HoneyDB network consists of globally deployed HoneyDB agents emulating various network services. The interactions with these services are what generates the data behind HoneyDB’s API. Data from a subset of these services is analyzed to produce the data for the API endpoints covered in this blog series. To get the full list of these analyzed services use the services endpoint.
Query services example:
Endpoint: /api/payload-history/services
curl --header "X-HoneyDb-ApiId: <api_id>" \
--header "X-HoneyDb-ApiKey: <api_key>" \
https://honeydb.io/api/payload-history/servicesThe response is a JSON array, example:
[
"CLICKHOUSEHTTP",
"DNS",
"DNS.UDP",
"HTTP",
"HTTP.ALT",
"ECHO",
"ECHO.UDP",
"ELASTICSEARCH",
"FTP",
"KUBLETAPI",
"SIP",
"MEMCACHED",
"MODBUS",
"REDIS",
"SMTP",
"TELNET",
"TFTP",
"VNC",
"WEBLOGIC"
]Now that we know what services are available we can use service names to query for attributes associated with a specific service of interest.
Note, the HoneyDB agent’s service coverage will continue to expand so this services list will expand over time.
Querying by service name example
Endpoint: /api/payload-history/SERVICE_NAME
curl --header "X-HoneyDb-ApiId: <api_id>" \
--header "X-HoneyDb-ApiKey: <api_key>" \
https://honeydb.io/api/payload-history/REDISIn this call to the API we can filter for the service by simply using the service name at the end of the URL path.
Example results
[
{
"date": "2025-01-05",
"service": "REDIS",
"attribute": "command",
"hash": "056c21b753f9d36caeccc3e6d05004a2",
"value": "info"
},
{
"date": "2025-01-05",
"service": "REDIS",
"attribute": "command",
"hash": "061a2b0a815d9e8e94de1c6c58454e09",
"value": "USER anonymous"
},
]This should be a familiar response structure and data (covered in part one of this series).
Attributes
With the attributes endpoint you can follow the same pattern as with services, query for attribute names, and then use attribute names to query for more detail. The first attributes endpoint returns the list of all attribute names that are available. The second endpoint will return all values with hashes for a given attribute name.
Query attributes example:
Endpoint: /api/payload-history/attributes
curl --header "X-HoneyDb-ApiId: <api_id>" \
--header "X-HoneyDb-ApiKey: <api_key>" \
https://honeydb.io/api/payload-history/attributesExample results:
[
"domain",
"scheme",
"host",
"path",
"http-version",
"user-agent",
"content-type",
"header",
"method",
"client-domain",
"sender-email",
"recipient-email",
"verify-email",
"shell-command",
"username_or_password",
"remote-frame-buffer",
"shell-command",
...
]A JSON array of all attribute names is returned. Now that you have the complete list of available attributes, we can query for attributes by name to list all values for an attribute.
Query by attribute name example:
Endpoint: /api/payload-history/attributes/ATTRIBUTE_NAME
curl --header "X-HoneyDb-ApiId: <api_id>" \
--header "X-HoneyDb-ApiKey: <api_key>" \
https://honeydb.io/api/payload-history/attributes/user-agent Example results:
[
{
"hash": "a5ecd7e3ca14fcb38a69343d1d47985a",
"value": "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"
},
{
"hash": "1cba7d39a7293c7e1de4e3333c3242f4",
"value": "Mozilla/3.0 (compatible; Indy Library)"
},
{
"hash": "55f9ce2b352fbe0f623b0a9513252de5",
"value": "Mozilla/4.0 (compatible; MSIE 4.01; Mac_PowerPC)"
},
{
"hash": "080ce9f58739390cf4d36e1e3b8eab1d",
"value": "Opera/6.x (Linux 2.4.8-26mdk i686; U) [en]"
},
...
]The response data is a JSON array of elements with each element containing only two fields. First is the MD5 hash of the attribute value, and then the value itself.
Pivot Back to Hash Endpoint
As shown in the previous example, the results from attribute queries contain only the attribute values and the hash of those values. However, with the hash value of an attribute of interest, we can leverage the hash endpoint we explored in part one. By utilizing the hash endpoint, we can retrieve the complete history of the attribute, providing deeper insights into its usage patterns across the HoneyDB dataset.
As an example, let’s say we want to see the history of the user-agent string Mozilla/3.0 (compatible; Indy Library), which is found in the results above. Along with it we find the hash for this user-agent string, 1cba7d39a7293c7e1de4e3333c3242f4. Next we use that hash value to query the hash endpoint, which yields the following results:
[
...
{
"date": "2023-05-27",
"service": "HTTP",
"attribute": "user-agent",
"hash": "1cba7d39a7293c7e1de4e3333c3242f4",
"value": "Mozilla/3.0 (compatible; Indy Library)"
},
{
"date": "2023-08-06",
"service": "HTTP",
"attribute": "user-agent",
"hash": "1cba7d39a7293c7e1de4e3333c3242f4",
"value": "Mozilla/3.0 (compatible; Indy Library)"
},
{
"date": "2023-12-19",
"service": "WEBLOGIC",
"attribute": "user-agent",
"hash": "1cba7d39a7293c7e1de4e3333c3242f4",
"value": "Mozilla/3.0 (compatible; Indy Library)"
}
]The example results above is just a sample from the full response. By reviewing the full result set we’ll see activity with this user-agent string spanned from 2014 to 2023. In addition we can observer which services were associated with the use of this user-agent string.
Conclusion
In this blog post, we explored the HoneyDB Payload History API, focusing on how to query data by service name and attribute name. We learned how to retrieve a list of available service and attribute names, and then how to use those names to filter results and gain deeper insights. By combining these API queries with the hash endpoint explored in part one, you can perform comprehensive analysis and uncover valuable information about network traffic patterns and potential threats.
In the next and final post, we will continue our exploration by diving into the last set of Payload History API endpoints for remote hosts!
To get started with the HoneyDB Payload History API, register for a community tier account at honeydb.io and generate your API keys. Note that the community tier is intended for personal and enterprise network defense (non-commercial use). If you have a commercial use case for this data as part of a product or service, commercial plans are available at https://honeydb.io/plans.
Stay tuned for the final installment to learn about the endpoints that uncover the remote IP addresses that are the source of all payload data in the HoneyDB Payload History API.