You can create a new topic by clicking the Create Topic button at the top right of the topic list.
All basic topic settings are available, and all detailed settings can be changed after the topic is created. The settings that are available when you create a new topic are listed below (Advanced settings are only available when editing a topic).
Topic settings configuration recommendations
You can easily configure settings for the new topic. You can cancel application of settings using the Reset default button.
- Settings specific to replication
- Storage optimized replication-factor 1 / minISR 1
- Neutral replication-factor 2 / minISR 1
- Max reliability replication-factor 3 / minISR 2
- Max availability replication-factor 3 / minISR 1 / Unclean leader election TRUE
- Settings specific to partitioning
- Topic-wide message order partition=1
- Minimum balancing partitions = brokers
- Parallelism balancing partitions = 3x brokers
- Settings specific to storage space management
- Temporary keep delete policy / keep 7days
- Keep last value compact policy
Basic setting
These are the most basic settings for a new topic.
- Topic name. No duplicates allowed.
- Partitions. Number of partitions in topic.
- Replication Factor. Number of replicas of the topic partition. Values larger than the current number of brokers are not allowed.
Frequently used settings
In general, settings that must be checked when creating a topic are included during the create process. By contrast, settings that are only changed occasionally are available when editing a topic (see Advanced settings).
- Minimum insync replicas. Minimum number of synchronized replicas. Must be less than replication_factor value.
- Clean-up Policy. Message log deletion policy: delete, compact, or both.
- Time to retain data. Maximum log retention period. Can be applied in conjunction with retention_bytes.
- Maximum message size. Maximum message size (after compression, if compression is applied).
- Segment time. Controls the cycle at which Kafka forces log rolling.
Advanced settings
When editing a topic, you can display or hide detailed settings using the Advanced Mode button at the bottom of the topic setting screen.
Replication
- Minimum insync replicas. Minimum number of synchronized replicas. Must be less than replication_factor value.
- Throttled replicas (leader side). List of replicas whose log replication needs to be regulated on the leader side.
- Throttled replicas (follower side). List of replicas whose log replication should be regulated on the follower side.
Manage Disk
- Clean-up Policy. Message log deletion policy (delete, compact, or both)
- Data storage size (Bytes to retain). Maximum log size. Data retention time and overlap can be applied.
- Time to retain data. Minimum retention period for message logs. Data storage size and overlap can be applied.
If cleanup and retention policies are not set correctly, disk usage can increase and the broker may crash.
Disk Cleanup Policy: Compaction options
- How long to keep a tombstone marker. How long to hold tombstone markers for compressed topic logs.
- Maximum time to keep ineligible messages before compression. Maximum time during which messages cannot be compressed from the log. Applies only to logs that are compressed.
- Minimum time to keep ineligible messages before compression. Minimum time a message will remain uncompressed in the log. Applies only to logs that are compressed.
- Cleaning frequency. Controls how often the log compressor attempts to prune logs. By default, if more than 50% of a log is compressed, it is not cleaned up.
Message Settings
- Maximum message size. Maximum message size (after compression, if compression is applied).
- Message format version. The message format version that the broker will use to add messages to the log. Generally same as Kafka broker version.
- Message timestamp type. Default timestamp to use if there is no timestamp field.
- Timestamp maximum time interval. The maximum time interval allowed between the timestamp when the broker receives a message and the timestamp specified in the message.
- Down-conversion of message. Controls whether down-conversion of message formats is used to satisfy consumer requests.
- Compression type. A way to compress and store messages. Typically selected by producer.
Flush
- Flush messages. Specifies the interval to force fsync of data written to the log.
- Message flush time (Time to flush message). Specifies the time interval to force fsync of data written to the log.
Segment
- Maximum random jitter. Maximum random jitter value to be subtracted from the reserved segment rolling time to avoid load when rolling segments.
- Segment file size. Control segment file size in logs.
- Segment time. Controls the cycle at which Kafka forces log rolling.
- Segment index. Controls the size of the index that maps offsets to file positions.
- File deletion time (Time to delete file). How long to wait before deleting a file from the file system.
- Pre-allocate new segments. Pre-allocate files on disk when creating a new log segment.
Individual topic partitions are stored and managed in multiple segment files for convenience.
Others
- Leader Selection Method (Unclean leader selection). If there is no broker with the latest data, a setting to determine whether to select a leader among OSR (Out-of-Sync Replicas), even if it does not have the latest data.
- Offset interval. Controls how often index entries are added to the offset index.