What Are the Capabilities and Features of Echosec's Data Providers?

Each of Echosec's 18 data providers impose unique limits on search parameters, post-return volume per search, and data types returned.

Types of data provider parameters: 

Search Parameter Limitations

Some data providers do not allow searching by all the parameters available in Echosec (location, keyword, date, and username). Supported search parameters are denoted in the provider drop-down on the Echosec Quick Search bar next to each provider name. These limitations are also described in the graphs shown below under "search types supported."

Geographical Area Limitations

Some providers that allow searching by post location will restrict the geographical area that can be queried per search. Reddit also limits the number of geographic subreddits that can be included in a search.

Blacklisted Keywords

If your keyword search doesn’t return any results, it’s possible that this keyword has been blacklisted by one or more data providers. Blacklisting keywords helps prevent searching or targeting individuals based on demographics including religion, race, or sexual orientation.

It is strictly against our Acceptable Use Policy to perform searches targeting specific demographics. Users will be notified if one of their searches is deemed unacceptable for these reasons, but Echosec Systems does not provide a user blacklist due to its fluidity.

Post Return Limits

Each data provider limits the number of posts returned for each Quick Search. These limits vary by provider from 10 to 1000+ results per search. This limitation is described in the graphs shown below under "post limit per request."

Historical Post Return Limits

Echosec Quick Searches return historical results until the post return limit described above is reached. Some providers also have limits with how far back they will return results. For example, one provider might return historical posts as far back as possible until its post return limit is reached, while another might only return posts published up to five days in the past. This limitation is described under "historical post return limit" in the graphs below.

Parameters by Data Provider

Below, find descriptions for each data provider's search parameters. Navigate to Echosec data providers by category:

Microblogging and Social Networks

Photo and Video Sharing Sites

Imageboards and Online Discussions/Chats

Social Journalism and News

Other

Microblogging and Social Networks


Provider Search types supported Content types returned Post limit per request Historical post return limit
OK.ru Keyword Text
Image
Video
100 2010-present
Mastodon Keyword
User
Text
Image
Video
200 3 months
Twitter Location
Keyword
User
Date
Text
Image
Video
100 All of time
Twitter Firehose Location
Keyword
User
Date
Text
Image
Video
500 All of time
VK Location
Keyword
User
Image 1000+ All of time
Tumblr Keyword Text
Image
100 5 days
Gab

Keyword
User

Text
Image
Video
200 4 months

 

Additional Notes

OK.ru

  • OK.ru returns results from tagged search endpoints.
  • OK.ru results may separately include individuals images and image albums. Albums containing a single image are automatically hidden to avoid showing duplicate content. Re-uploaded OK.ru user posts may still result in duplicate content within Echosec.
Twitter
  • For Pro users, author search is only available if the user integrates a Twitter account.
  • For Enterprise users, author search will only return results for one month at a time. If a user integrates their Twitter account, this limitation is removed and 200 results will be returned.
  • Retweets are excluded from results, except when searching by author.
  • Location searches return tweets within a radius of the given latitude/longitude. In order of preference, Twitter returns locations by precisely located tweets, then locations from Twitter's Geotagging API, and lastly by the user's profile location. This means that tweets aren't always precisely located, especially if the user's profile location is made up or inaccurate.
  • Twitter author searches query a different endpoint than Twitter location or keyword searches. This means that author searches have different conditions for returning imprecise geotagged results. For example, say you run a location search for Victoria and see a result that isn't precisely geotagged. If you run an author search for that post's author over Victoria, you will not see that post in the results. This is because the author search endpoint does not return imprecise geotagged posts.
  • User searches do not support date parameters.
  • User searches interface with location and keyword searches, and return the last 200 tweets from a specific user.
  • There are no area limitations for Twitter—however, smaller areas return more precise results.
  • Twitter supports emoji and other UTF-8 characters.

Twitter Firehose

  • For Enterprise users, author search will only return results for one month at a time. If a user integrates their Twitter account, this limitation is removed and 500 results will be returned.
  • Retweets are excluded from results, except when searching by author.
  • Users with Twitter Firehose access have access to a tweet live stream feature for Saved Searches.
  • Limitations on searches that return higher-than-normal post volumes are implemented on a case by case basis.
VK
  • VK returns precisely geo-tagged posts.
  • Keyword searches support emoji and other UTF-8 characters.
  • User searches are not supported from the Search Bar. Geo and keyword search results do not contain poster info, but user search is supported in a post's detail view. This returns profile and wall photos.
Tumblr
  • Medium, Pinterest, and Tumblr combined share a maximum of 100 results.
  • Echosec searches article title text.
  • .gif Files are not considered videos by Echosec's algorithm.

Photo and Video Sharing Sites

 

Provider Search types supported Content types returned Post limit per request Historical post time limit per Quick Search 
Flickr Location
Keyword
User
Date
Image
Video
250 (Location)
500 (Keyword)
500 (User)
All of time
Vimeo Keyword Video 50 1 day
YouTube Location
Keyword
User
Video 200 All of time

 

Additional Notes

Flickr

  • Location results are precisely geotagged. 
  • Photo and video titles, descriptions, and tags containing the relevant keywords are returned.
  • Flickr searches support emoji and other UTF-8 characters.

Vimeo

  • Searches support emoji and other UTF-8 characters.

YouTube

  • Location results are precisely geotagged.
  • Searches support emoji and other UTF-8 characters.
  • The most reliable method of searching for YouTube users is via the user icon in the post-detail view. User search is available from the Quick Search bar, but is restricted to the channel owner's username. Functionality to search channel names is in development.

Imageboards and Online Discussions/Chats

 

Provider Search types supported Content types returned Post limit per request Historical post time limit per Quick Search 
4chan Keyword Text 200 ~November 2019-present
Reddit Location
Keyword
User
Text
Image
Video
100 All of time
Discord Keyword
User
Text 10 threads 1 year
Telegram Keyword
User
Text 10 threads 1 year
Raddle.me Keyword
User
Text 200 All of time

 

Additional Notes


Reddit

  • Location results are predicted based on their subreddit, which is correlated with our proprietary geospatial datastore.

Discord

  • The post limit per request (10 threads) is combined with Telegram results returned.

Telegram

  • The post limit per request (10 threads) is combined with Discord results returned.

Social Journalism and News

 

Provider Search types supported Content types returned Post limit per request Historical post time limit per Quick Search 
Medium Keyword Text
Image
100 5 days
Bing News Keyword Text 10 5 days
Webhose News Keyword Text 100 5 days

 

      Additional Notes

      Medium

      • Medium, Pinterest & Tumblr combined share a maximum of 100 results.
      Bing News
      • Searches the web for news articles with Bing News Search.
      • Searches support emojis and other UTF-8.
      Webhose News
      • Searches the web for news articles with Webhose.io News Search.

      Other

       

      Provider Search types supported Content types returned Post limit per request Historical post time limit per Quick Search 
      Pinterest Keyword Text 100 4 days

       

      Additional Notes

      Pinterest

      • Searches search Pin title text.
      • Medium, Pinterest & Tumblr combined share a maximum of 100 results.
                        More info about data providers, please!

                        Want to learn more about our data providers? Find data provider descriptions and statistics here.