Class: StructuredDataToSql::Json::Profiles::KhorosApiExport

Inherits:
Object
  • Object
show all
Defined in:
lib/structured_data_to_sql/json/profiles/khoros_api_export.rb

Overview

Appends converter-ready Khoros API helper tables without changing the append-only raw tables. An occurrence is identified by its envelope id and every flattened context field; the greatest raw _sid wins as a whole (the exporter contract: append-only, last occurrence wins).

Defined Under Namespace

Classes: InputValidator

Constant Summary collapse

NAME = 
:khoros_api_export
VERSION =

Bumped in lockstep with PROFILE_VERSION in the Khoros converter's KhorosApiExportContentStaging whenever a canonical table's shape or semantics change. Version 2 adds normalized root-message read-only state.

2
PREFIX = 
"discourse_khoros_api"
METADATA_TABLE = 
"_structured_data_to_sql_profiles"
TABLE_OPTIONS =

Explicit storage options on every canonical table so they never inherit database defaults: utf8mb4 VARCHAR(255) keys need DYNAMIC row format (a 1020-byte index fails on COMPACT/REDUNDANT servers).

"ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci ROW_FORMAT=DYNAMIC"
REQUIRED_SCHEMA = 
{
  "users" => %w[_sid id data_id data_login],
  "messages" => %w[
    _sid
    id
    data_id
    data_author_id
    data_board_id
    data_body
    data_conversation_id
    data_depth
    data_post_time
    data_read_only
    data_subject
  ],
  "nodes" => %w[_sid id data_depth data_id data_node_type data_title]
}.freeze
TABLES = 
%w[
  profiles
  avatars
  users
  nodes
  ranks
  roles
  memberships
  badge_definitions
  badge_grants
  engagement
  floated_messages
  tags
  images
  attachments
  files
  source_messages
].map { |name| "#{PREFIX}_#{name}" }.freeze
USER_STRING_FIELDS = 
{
  id: %w[data_id],
  login: %w[data_login],
  email: %w[data_email data_email_address],
  sso_id: %w[data_sso_id data_ssoid data_sso_uid],
  rank_id: %w[data_rank_id],
  first_name: %w[data_first_name],
  last_name: %w[data_last_name],
  biography: %w[data_biography],
  location: %w[data_location],
  web_page_url: %w[data_web_page_url data_url],
  language: %w[data_language],
  title: %w[data_title],
  signature: %w[data_signature],
  inline_avatar_url: %w[
    data_avatar_profile
    data_avatar_inline
    data_avatar_message
  ],
  registration_status: %w[data_registration_data_status],
  timezone: %w[data_config_timezone data_timezone],
  profile_privacy: %w[
    data_personal_info_privacy
    data_profile_privacy
    data_profile_properties_personal_info_privacy
  ],
  online_status_privacy: %w[
    data_online_status_privacy
    data_profile_properties_online_status_privacy
  ]
}.freeze
USER_SCALAR_FIELDS = 
{
  registration_time: %w[data_registration_data_registration_time],
  last_visit_time: %w[data_last_visit_time],
  banned: %w[data_banned],
  approved: %w[data_email_verified],
  deleted: %w[data_deleted]
}.freeze
USER_CANONICAL_COLUMNS = 
(USER_STRING_FIELDS.values + USER_SCALAR_FIELDS.values)
.flatten
.to_set
.freeze
USER_DELEGATED_COLUMN_PATTERNS = 
[
  /\Adata_user_badges(?:_|\z)/,
  /\Adata_registration_data_sso_registration_fields(?:_|\z)/
].freeze
USER_INTENTIONAL_RAW_ONLY_COLUMNS =

Envelope, transport, link, and type columns plus v2 presentation fields that carry no member attribute. Retained raw, never warned.

%w[
  _sid
  id
  api
  exported_at
  export_reason
  url
  data_href
  data_type
  data_view_href
  data_avatar_favicon
  data_avatar_print
  data_avatar_type
  data_date_pattern
  data_friendly_date_enabled
  data_friendly_date_max_age
  data_mailbox_type
  data_metrics_type
  data_rank_type
  data_registration_data_confirm_email_status
  data_registration_data_type
].to_set.freeze
USER_INTENTIONAL_RAW_ONLY_COLUMN_PATTERNS =

Lazy v2 collection stubs ("SELECT ...") and the rank definition denormalized onto the user row (canonical via ranks).

[
  /\Adata_[a-z0-9_]+_query\z/,
  /\Adata_rank_(?!id\z)/
].freeze
USER_REVIEW_ONLY_COLUMNS =

Real per-member values the converter does not consume yet; warned once per run so a migration decision stays visible.

%w[
  data_email_address_privacy
  data_online_status
  data_registration_data_registration_access_level
  data_remember_password
  data_show_user_signatures
].to_set.freeze
USER_REVIEW_ONLY_COLUMN_PATTERNS = 
[
  /\Adata_metrics_/,
  /\Adata_notes(?:_|\z)/,
  /\Adata_c_/
].freeze

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(table_columns) ⇒ KhorosApiExport

Returns a new instance of KhorosApiExport.



880
881
882
883
 
# File 'lib/structured_data_to_sql/json/profiles/khoros_api_export.rb', line 880

def initialize(table_columns)
  @table_columns =
    table_columns.transform_values { |value| Array(value).map(&:to_s) }
end

Class Method Details

.write_preamble(out) ⇒ Object 



876
877
878
 
# File 'lib/structured_data_to_sql/json/profiles/khoros_api_export.rb', line 876

def self.write_preamble(out)
  out.write("DROP TABLE IF EXISTS `#{METADATA_TABLE}`;\n")
end

Instance Method Details

#validate! ⇒ Object

Raises:



912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
 
# File 'lib/structured_data_to_sql/json/profiles/khoros_api_export.rb', line 912

def validate!
  problems =
    REQUIRED_SCHEMA.filter_map do |table, required|
      next "#{table}=missing" unless table_exists?(table)

      missing = required - columns(table)
      "#{table}.#{missing.join(", #{table}.")}=missing" if missing.any?
    end
  return if problems.empty?

  raise UsageError,
        "Unsupported khoros_api_export schema: #{problems.join("; ")}. " \
          "Required collections are users, messages, and nodes; a message_authors-only export is not supported. " \
          "Regenerate the API export with the current khoros-exporter and prepare it again."
end

#write(out) ⇒ Object 



885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
 
# File 'lib/structured_data_to_sql/json/profiles/khoros_api_export.rb', line 885

def write(out)
  validate!
  @out = out
  out.write(
    "\n-- structured_data_to_sql profile: khoros_api_export v#{VERSION}\n"
  )
  TABLES.each { |table| statement("DROP TABLE IF EXISTS `#{table}`") }
  create_profiles
  create_avatars
  create_users
  create_nodes
  create_ranks
  create_roles
  create_memberships
  create_badges
  create_engagement
  create_floated_messages
  create_tags
  create_images
  create_attachments
  create_files
  create_source_messages
  create_metadata
ensure
  @out = nil
end