Class: XTwitterScraper::Resources::Extractions

Inherits:
Object
  • Object
show all
Defined in:
lib/x_twitter_scraper/resources/extractions.rb,
sig/x_twitter_scraper/resources/extractions.rbs

Overview

Bulk data extraction (23 tool types)

Instance Method Summary collapse

Constructor Details

#initialize(client:) ⇒ Extractions

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns a new instance of Extractions.

Parameters:



297
298
299
 
# File 'lib/x_twitter_scraper/resources/extractions.rb', line 297

def initialize(client:)
  @client = client
end

Instance Method Details

#estimate_cost(tool_type:, advanced_query: nil, any_words: nil, bounding_box: nil, cashtags: nil, conversation_id: nil, exact_phrase: nil, exclude_words: nil, from_user: nil, hashtags: nil, in_reply_to_tweet_id: nil, language: nil, list_id: nil, media_type: nil, mentioning: nil, min_faves: nil, min_quotes: nil, min_replies: nil, min_retweets: nil, place: nil, place_country: nil, point_radius: nil, quotes: nil, quotes_of_tweet_id: nil, replies: nil, results_limit: nil, retweets: nil, retweets_of_tweet_id: nil, search_query: nil, since_date: nil, target_community_id: nil, target_list_id: nil, target_space_id: nil, target_tweet_id: nil, target_username: nil, to_user: nil, until_date: nil, url: nil, verified_only: nil, request_options: {}) ⇒ XTwitterScraper::Models::ExtractionEstimateCostResponse

Some parameter documentations has been truncated, see Models::ExtractionEstimateCostParams for more details.

Estimate extraction cost

Parameters:

  • tool_type (Symbol, XTwitterScraper::Models::ExtractionEstimateCostParams::ToolType) —

    Identifier for the extraction tool used to run a job.

  • advanced_query (String) —

    Raw advanced query string appended to the estimate (tweet_search_extractor)

  • any_words (String) —

    Alternative words or quoted phrases for estimated results. Separate with spaces,

  • bounding_box (String) —

    Geo bounding box used for estimation, e.g. -74.1 40.6 -73.9 40.8 (tweet_search_e

  • cashtags (String) —

    Cashtags applied to the estimate, separated by spaces, commas, or lines.

  • conversation_id (String) —

    Conversation ID filter used for estimation (tweet_search_extractor)

  • exact_phrase (String) —

    Exact phrase filter for search estimation

  • exclude_words (String) —

    Words or quoted phrases excluded from estimated results. Separate with spaces, c

  • from_user (String) —

    Estimate only tweets from this author username (tweet_search_extractor)

  • hashtags (String) —

    Hashtags applied to the estimate, separated by spaces, commas, or lines.

  • in_reply_to_tweet_id (String) —

    Estimate only replies to this tweet ID (tweet_search_extractor)

  • language (String) —

    Language code used for estimate filtering (tweet_search_extractor)

  • list_id (String) —

    Estimate search results within this list ID (tweet_search_extractor)

  • media_type (Symbol, XTwitterScraper::Models::ExtractionEstimateCostParams::MediaType) —

    Media type used for estimate filtering (tweet_search_extractor)

  • mentioning (String) —

    Estimate tweets mentioning this username (tweet_search_extractor)

  • min_faves (Integer) —

    Minimum likes threshold for estimated results (tweet_search_extractor)

  • min_quotes (Integer) —

    Minimum quote count threshold for estimated results (tweet_search_extractor)

  • min_replies (Integer) —

    Minimum replies threshold for estimated results (tweet_search_extractor)

  • min_retweets (Integer) —

    Minimum retweets threshold for estimated results (tweet_search_extractor)

  • place (String) —

    Estimate search results within this place ID (tweet_search_extractor)

  • place_country (String) —

    Estimate search results within this country code (tweet_search_extractor)

  • point_radius (String) —

    Geo point radius used for estimation, e.g. -73.99 40.73 25mi (tweet_search_extra

  • quotes (Symbol, XTwitterScraper::Models::ExtractionEstimateCostParams::Quotes) —

    Quote mode used for estimation (tweet_search_extractor)

  • quotes_of_tweet_id (String) —

    Estimate only quotes of this tweet ID (tweet_search_extractor)

  • replies (Symbol, XTwitterScraper::Models::ExtractionEstimateCostParams::Replies) —

    Reply mode used for estimation (tweet_search_extractor)

  • results_limit (Integer) —

    Maximum number of results to estimate. When set, the estimate caps projected res

  • retweets (Symbol, XTwitterScraper::Models::ExtractionEstimateCostParams::Retweets) —

    Retweet mode used for estimation (tweet_search_extractor)

  • retweets_of_tweet_id (String) —

    Estimate only retweets of this tweet ID (tweet_search_extractor)

  • search_query (String) —

    Required for tweet_search_extractor & community_search.

  • since_date (Date) —

    Estimate start date in YYYY-MM-DD format (tweet_search_extractor)

  • target_community_id (String) —

    Required for community_post_extractor & community_search.

  • target_list_id (String) —

    Required for list_follower_explorer, list_member_extractor & list_post_extractor

  • target_space_id (String) —

    Required for space_explorer.

  • target_tweet_id (String)
  • target_username (String)
  • to_user (String) —

    Estimate replies sent to this username (tweet_search_extractor)

  • until_date (Date) —

    Estimate end date in YYYY-MM-DD format (tweet_search_extractor)

  • url (String) —

    URL substring or domain filter used for estimation (tweet_search_extractor)

  • verified_only (Boolean) —

    Estimate only verified authors (tweet_search_extractor)

  • request_options (XTwitterScraper::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



156
157
158
159
160
161
162
163
164
165
 
# File 'lib/x_twitter_scraper/resources/extractions.rb', line 156

def estimate_cost(params)
  parsed, options = XTwitterScraper::ExtractionEstimateCostParams.dump_request(params)
  @client.request(
    method: :post,
    path: "extractions/estimate",
    body: parsed,
    model: XTwitterScraper::Models::ExtractionEstimateCostResponse,
    options: options
  )
end

#export_results(id, format_:, request_options: {}) ⇒ StringIO

Export extraction results

Parameters:

Returns:

  • (StringIO)

See Also:



180
181
182
183
184
185
186
187
188
189
190
191
 
# File 'lib/x_twitter_scraper/resources/extractions.rb', line 180

def export_results(id, params)
  parsed, options = XTwitterScraper::ExtractionExportResultsParams.dump_request(params)
  query = XTwitterScraper::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: ["extractions/%1$s/export", id],
    query: query.transform_keys(format_: "format"),
    headers: {"accept" => "application/octet-stream"},
    model: StringIO,
    options: options
  )
end

#list(cursor: nil, limit: nil, status: nil, tool_type: nil, request_options: {}) ⇒ XTwitterScraper::Models::ExtractionListResponse

Some parameter documentations has been truncated, see Models::ExtractionListParams for more details.

List extraction jobs

Parameters:

Returns:

See Also:



54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/x_twitter_scraper/resources/extractions.rb', line 54

def list(params = {})
  parsed, options = XTwitterScraper::ExtractionListParams.dump_request(params)
  query = XTwitterScraper::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: "extractions",
    query: query.transform_keys(tool_type: "toolType"),
    model: XTwitterScraper::Models::ExtractionListResponse,
    options: options
  )
end

#retrieve(id, cursor: nil, limit: nil, request_options: {}) ⇒ XTwitterScraper::Models::ExtractionRetrieveResponse

Get extraction results

Parameters:

  • id (String) —

    Extraction public ID (UUID)

  • cursor (String) —

    Cursor for keyset pagination from prior response next_cursor

  • limit (Integer) —

    Maximum number of results to return (1-1000, default 100)

  • request_options (XTwitterScraper::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



22
23
24
25
26
27
28
29
30
31
32
 
# File 'lib/x_twitter_scraper/resources/extractions.rb', line 22

def retrieve(id, params = {})
  parsed, options = XTwitterScraper::ExtractionRetrieveParams.dump_request(params)
  query = XTwitterScraper::Internal::Util.encode_query_params(parsed)
  @client.request(
    method: :get,
    path: ["extractions/%1$s", id],
    query: query,
    model: XTwitterScraper::Models::ExtractionRetrieveResponse,
    options: options
  )
end

#run(tool_type:, advanced_query: nil, any_words: nil, bounding_box: nil, cashtags: nil, conversation_id: nil, exact_phrase: nil, exclude_words: nil, from_user: nil, hashtags: nil, in_reply_to_tweet_id: nil, language: nil, list_id: nil, media_type: nil, mentioning: nil, min_faves: nil, min_quotes: nil, min_replies: nil, min_retweets: nil, place: nil, place_country: nil, point_radius: nil, quotes: nil, quotes_of_tweet_id: nil, replies: nil, results_limit: nil, retweets: nil, retweets_of_tweet_id: nil, search_query: nil, since_date: nil, target_community_id: nil, target_list_id: nil, target_space_id: nil, target_tweet_id: nil, target_username: nil, to_user: nil, until_date: nil, url: nil, verified_only: nil, request_options: {}) ⇒ XTwitterScraper::Models::ExtractionRunResponse

Some parameter documentations has been truncated, see Models::ExtractionRunParams for more details.

Run extraction

Parameters:

  • tool_type (Symbol, XTwitterScraper::Models::ExtractionRunParams::ToolType) —

    Identifier for the extraction tool used to run a job.

  • advanced_query (String) —

    Raw advanced search query appended as-is (tweet_search_extractor)

  • any_words (String) —

    Words or quoted phrases where any one can match. Separate with spaces, commas, o

  • bounding_box (String) —

    Geo bounding box, e.g. -74.1 40.6 -73.9 40.8 (tweet_search_extractor)

  • cashtags (String) —

    Cashtags separated by spaces, commas, or lines. (tweet_search_extractor)

  • conversation_id (String) —

    Conversation ID filter (tweet_search_extractor)

  • exact_phrase (String) —

    Exact phrase to match (tweet_search_extractor)

  • exclude_words (String) —

    Words or quoted phrases to exclude. Separate with spaces, commas, or lines. (twe

  • from_user (String) —

    Filter by author username (tweet_search_extractor)

  • hashtags (String) —

    Hashtags separated by spaces, commas, or lines. (tweet_search_extractor)

  • in_reply_to_tweet_id (String) —

    Only replies to this tweet ID (tweet_search_extractor)

  • language (String) —

    Language code filter (tweet_search_extractor)

  • list_id (String) —

    Search within a list ID (tweet_search_extractor)

  • media_type (Symbol, XTwitterScraper::Models::ExtractionRunParams::MediaType) —

    Media type filter (tweet_search_extractor)

  • mentioning (String) —

    Filter tweets mentioning a username (tweet_search_extractor)

  • min_faves (Integer) —

    Minimum likes threshold (tweet_search_extractor)

  • min_quotes (Integer) —

    Minimum quote count threshold (tweet_search_extractor)

  • min_replies (Integer) —

    Minimum replies threshold (tweet_search_extractor)

  • min_retweets (Integer) —

    Minimum retweets threshold (tweet_search_extractor)

  • place (String) —

    Search within a place ID (tweet_search_extractor)

  • place_country (String) —

    Search within a country code (tweet_search_extractor)

  • point_radius (String) —

    Geo point radius, e.g. -73.99 40.73 25mi (tweet_search_extractor)

  • quotes (Symbol, XTwitterScraper::Models::ExtractionRunParams::Quotes) —

    Quote mode (tweet_search_extractor)

  • quotes_of_tweet_id (String) —

    Only quotes of this tweet ID (tweet_search_extractor)

  • replies (Symbol, XTwitterScraper::Models::ExtractionRunParams::Replies) —

    Reply mode (tweet_search_extractor)

  • results_limit (Integer) —

    Maximum number of results to extract. When set, the extraction stops after reach

  • retweets (Symbol, XTwitterScraper::Models::ExtractionRunParams::Retweets) —

    Retweet mode (tweet_search_extractor)

  • retweets_of_tweet_id (String) —

    Only retweets of this tweet ID (tweet_search_extractor)

  • search_query (String) —

    Required for tweet_search_extractor & community_search.

  • since_date (Date) —

    Start date YYYY-MM-DD (tweet_search_extractor)

  • target_community_id (String) —

    Required for community_post_extractor & community_search.

  • target_list_id (String) —

    Required for list_follower_explorer, list_member_extractor & list_post_extractor

  • target_space_id (String) —

    Required for space_explorer.

  • target_tweet_id (String)
  • target_username (String)
  • to_user (String) —

    Filter replies sent to a username (tweet_search_extractor)

  • until_date (Date) —

    End date YYYY-MM-DD (tweet_search_extractor)

  • url (String) —

    URL substring or domain filter (tweet_search_extractor)

  • verified_only (Boolean) —

    Only verified authors (tweet_search_extractor)

  • request_options (XTwitterScraper::RequestOptions, Hash{Symbol=>Object}, nil)

Returns:

See Also:



283
284
285
286
287
288
289
290
291
292
 
# File 'lib/x_twitter_scraper/resources/extractions.rb', line 283

def run(params)
  parsed, options = XTwitterScraper::ExtractionRunParams.dump_request(params)
  @client.request(
    method: :post,
    path: "extractions",
    body: parsed,
    model: XTwitterScraper::Models::ExtractionRunResponse,
    options: options
  )
end