update kafka instructions for dealing with messages etc #10171
Conversation
product-auto-label
bot
added
samples
| @@ -200,6 +212,41 @@ gcloud compute instances create-with-container kafka-vm \ | |||
| --tags "kafka-server" | |||
| ``` | |||
| ### Sending messages to Kafka server | |||
| The Kafka server should be running at this point, but in its current state no | |||
There was a problem hiding this comment.
nit: blending in context with instruction to ssh makes this a little difficult to parse.
| the template is started will be present when the template is started. If the | ||
| desire is to send messages after the template has started, then the messages | ||
| will be processed as they are sent. | ||
There was a problem hiding this comment.
I'd restructure the following menu of items into a recipe to follow. (deletion probably at the bottom)
There was a problem hiding this comment.
Suggested ordering:
Pre-Req SSH into the Kafka VM
$ gcloud compute ssh kafka-vm --zone "$ZONE"1. Create a Topic
Replace with your topic ID.
(show docker command)
2. Send Messages to the Topic
Run the console producer to send messages. After running the command, type a message and press Enter. You can send multiple messages. Press Ctrl+C to stop the producer.
Note: You can run this step either before starting the Dataflow template (messages will be ready) or while it's running (messages will be processed as they arrive).
(show docker command)
3. (Optional) Verify the Messages
You can check that your messages were sent correctly by starting a consumer. This will print all messages from the beginning of the topic. Press Ctrl+C to exit.
4. (Optional) Delete a Topic
| @@ -183,9 +183,21 @@ To learn more about pricing, see the | |||
| ```sh | |||
| export KAFKA_IMAGE="gcr.io/$PROJECT/samples/dataflow/kafka:latest" | |||
| # Note: If the project name has `:` in it that signifies a project within an | |||
| # organization (e.g. `example.com:project-id`), replace those with `/` so that | |||
| # the Kafa image can be found appropriately. | |||
| @@ -33,7 +33,7 @@ ENV ZOOKEEPER_PORT="${ZOOKEEPER_PORT:-2181}" | |||
|
|
|||
| # Download and install Apache Kafka. | |||
| RUN apk add --no-cache bash \ | |||
| && wget http://apache.mirrors.spacedump.net/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz \ | |||
| && wget http://archive.apache.org/dist/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz \ | |||
There was a problem hiding this comment.
probably should use the https:// here
Description
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist
pom.xmlparent set to latestshared-configurationmvn clean verifyrequiredmvn -P lint checkstyle:checkrequiredmvn -P lint clean compile pmd:cpd-check spotbugs:checkadvisory only