Configure Space tools

This page describes a proposed Kafka Improvement Proposal (KIP) process for proposing a major change to Kafka.

Getting Started

If this is your first time contributing:

1. Sign up for the Developer mailing list [email protected] . The instructions to sign up are here: http://kafka.apache.org/contact
2. Create a wiki ID ( https://cwiki.apache.org/confluence/signup.action self-service is currently not available: please request an manual account creating by leaving a comment on INFRA-25451 Getting issue details... STATUS until the issue is resolved)
3. Create a Jira ID (It's a different system than the wiki) using the ASF Self-serve Portal ( https://selfserve.apache.org/jira-account.html )
4. Send an email to the dev mailing list ( [email protected] ) containing your wiki ID and Jira ID requesting permissions to contribute to Apache Kafka.

Purpose

We want to make Kafka a core architectural component for users. We also support a large number of integrations with other tools, systems, and clients. Keeping this kind of usage health requires a high level of compatibility between releases — core architectural elements can't break compatibility or shift functionality from release to release. As a result each new major feature or public api has to be done in a way that we can stick with it going forward.

This means when making this kind of change we need to think through what we are doing as best we can prior to release. And as we go forward we need to stick to our decisions as much as possible. All technical decisions have pros and cons so it is important we capture the thought process that lead to a decision or design to avoid flip-flopping needlessly.

Hopefully we can make these proportional in effort to their magnitude — small changes should just need a couple brief paragraphs, whereas large changes need detailed design discussions.

This process also isn't meant to discourage incompatible changes — proposing an incompatible change is totally legitimate. Sometimes we will have made a mistake and the best path forward is a clean break that cleans things up and gives us a good foundation going forward. Rather this is intended to avoid accidentally introducing half thought-out interfaces and protocols that cause needless heartburn when changed. Likewise the definition of "compatible" is itself squishy: small details like which errors are thrown when are clearly part of the contract but may need to change in some circumstances, likewise performance isn't part of the public contract but dramatic changes may break use cases. So we just need to use good judgement about how big the impact of an incompatibility will be and how big the payoff is.

What is considered a "major change" that needs a KIP?

Any of the following should be considered a major change:

• Any major new feature, subsystem, or piece of functionality
• Any change that impacts the public interfaces of the project

What are the "public interfaces" of the project?

All of the following are public interfaces that people build around:

• Binary log format
• The network protocol and api behavior
• Any class for which the build generates Javadoc.  This includes (but is not limited to; look for javadoc {} sections in build.gradle for the complete list):
  

  • org/apache/kafka/common/serialization

  • org/apache/kafka/common

  • org/apache/kafka/common/errors

  • org/apache/kafka/clients/producer

  • org/apache/kafka/clients/consumer (eventually, once stable)

• Configuration, especially client configuration
• Monitoring
• Command line tools and arguments

Not all compatibility commitments are the same. We need to spend significantly more time on log format and protocol as these break code in lots of clients, cause downtime releases, etc. Public apis are next as they cause people to rebuild code and lead to compatibility issues in large multi-dependency projects (which end up requiring multiple incompatible versions). Configuration, monitoring, and command line tools can be faster and looser — changes here will break monitoring dashboards and require a bit of care during upgrades but aren't a huge burden.

For the most part monitoring, command line tool changes, and configs are added with new features so these can be done with a single KIP.

What should be included in a KIP?

A KIP should contain the following sections:

• Motivation : describe the problem to be solved
• Proposed Change : describe the new thing you want to do. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences, depending on the scope of the change.
• New or Changed Public Interfaces : impact to any of the "compatibility commitments" described above. We want to call these out in particular so everyone thinks about them.
• Migration Plan and Compatibility : if this feature requires additional support for a no-downtime upgrade describe how that will work
• Rejected Alternatives : What are the other alternatives you considered and why are they worse? The goal of this section is to help people understand why this is the best solution now, and also to prevent churn in the future when old alternatives are reconsidered.

Who should initiate the KIP?

Anyone can initiate a KIP but you shouldn't do it unless you have an intention of getting the work done to implement it (otherwise it is silly).

Process

Here is the process for making a KIP:

1. Click Create KIP . Take the next available KIP number and give your proposal a descriptive heading. e.g. "KIP 42: Allow Infinite Retention With Bounded Disk Usage".
2. Update the next available KIP number below.
3. Fill in the sections as described above
4. Also add an entry to the table KIPs under discussion (for Streams API KIPs, please also add it to Kafka Streams sub page ).
5. Start a [DISCUSS] thread on the Apache mailing list. Please ensure that the subject of the thread is of the format [DISCUSS] KIP-{your KIP number} {your KIP heading} and the body contains a link to your new KIP. The discussion should happen on the mailing list, not on the wiki, since the wiki comment system doesn't work well for larger discussions. In the process of the discussion you may update the proposal. You should let people know the changes you are making. When you feel you have a finalized proposal
6. Once the proposal is finalized call a [VOTE] to have the proposal adopted. These proposals are more serious than code changes and more serious even than release votes. The criteria for acceptance is lazy majority . The vote should remain open for at least 72 hours.
7. Please update the KIP wiki page, and the index below, to reflect the current stage of the KIP after a vote. This acts as the permanent record indicating the result of the KIP (e.g., Accepted or Rejected). Also report the result of the KIP vote to the voting thread on the mailing list so the conclusion is clear.

KIP round-up

Next KIP Number: 1067

Use this number as the identifier for your KIP and increment this value.

A dopted KIPs

Please insert new rows in sorted order (descending by KIP number).

KIPs under discussion

Please insert new rows in sorted order (ascending by KIP number). Please move the adopted/accepted KIPs to the "Adopted KIPs" table above.

Go to link

Dormant/inactive KIPs

Please insert new rows in sorted order (ascending by KIP number).

Discarded KIPs

Please insert new rows in sorted order (ascending by KIP number).

KIP Discussion Recordings

Powered by Atlassian Confluence 7.19.20

Printed by Atlassian Confluence 7.19.20

Report a bug

Atlassian News