Back propagation of schemas

[rpatrick00]

I have written custom snaps that advertise their input and output schemas. If I place a mapper directly prior to one of my snaps, the mapper picks up and shows the schema, just as I would expect it to.

If I had to write another custom snap (let’s call it the “prune tree of nulls” snap) to place in between the mapper and my current custom snaps, is it possible for the “prune tree of nulls” snap to get access to the downstream schema advertised by my custom snap so that it can advertise the same schema? How does the mapper snap display the target schema? Is it possible for me to write a custom snap that does the same thing?

[ptaylor]

Try this in your snap class:

import com.snaplogic.api.SchemaPropagator;

add “implements SchemaPropagator.Passthrough” to your class

This interface functions as a “mixin” by relying on a feature that was new in Java 8: There are default implementations for the two methods defined by this interface that should do what you need if you’re looking to simply pass through the schema in both directions without modification. This is exactly what our Copy snap does.

Sorry, this is a relatively new feature of our SDK and it’s not documented in our developer docs yet. We appreciate that you’re building such a full-featured snap that does everything a good snap should. To make up for the lack of examples or javadocs for this class, I’ve included its source code below. You may notice an alternative to the “Passthrough” interface called “Original”, that gives you some clues about how you might alter the schema in either direction if you need to.

Hope that helps.

/*
 * SnapLogic - Data Integration
 *
 * Copyright (C) 2019, SnapLogic, Inc.  All rights reserved.
 *
 * This program is licensed under the terms of
 * the SnapLogic Commercial Subscription agreement.
 *
 * "SnapLogic" is a trademark of SnapLogic, Inc.
 */

package com.snaplogic.api;

import com.snaplogic.common.SnapType;
import com.snaplogic.snap.schema.api.ObjectSchema;
import com.snaplogic.snap.schema.api.Schema;
import com.snaplogic.snap.schema.api.SchemaProvider;

/**
 * Snaps should implement this interface if they wish to propagate schema from upstream snaps to
 * downstream snaps and vice-versa.
 */
public interface SchemaPropagator {
    /**
     * Mixin for snaps that directly pass documents without modification.
     */
    interface Passthrough extends SchemaPropagator {
        @Override
        default void pullDownstreamInputSchema(SchemaProvider schemaProvider, final String name,
                ObjectSchema downstreamSchema, String view, ObjectSchema inputSchema) {
            inputSchema.merge(downstreamSchema);
        }

        @Override
        default void propagateUpstreamOutputSchema(SchemaProvider schemaProvider, final String name,
                ObjectSchema upstreamSchema, String view, ObjectSchema outputSchema) {
            outputSchema.merge(upstreamSchema);
        }
    }

    /**
     * Mixin for snaps that pass their input documents to their outputs in the "original" property.
     */
    interface Original extends SchemaPropagator {
        @Override
        default void pullDownstreamInputSchema(SchemaProvider schemaProvider, final String name,
                ObjectSchema downstreamSchema, String view, ObjectSchema inputSchema) {
            Schema original = downstreamSchema.getChild("original");

            if (original != null) {
                inputSchema.merge(original);
            }
        }

        @Override
        default void propagateUpstreamOutputSchema(SchemaProvider schemaProvider, final String name,
                ObjectSchema upstreamSchema, String view, ObjectSchema outputSchema) {
            ObjectSchema originalSchema = schemaProvider.createSchema(SnapType.COMPOSITE,
                    "original");

            originalSchema.merge(upstreamSchema);
            outputSchema.addChild(originalSchema);
        }
    }

    /**
     * Snaps should implement this method to propagate schema from downstream snaps to this
     * snap's input views.
     *
     * @param schemaProvider
     * @param name The name of the output view that is connected to a downstream snap.
     * @param downstreamSchema The schema of the downstream input view.
     * @param view The name of this snap's input view that is being modified.
     * @param inputSchema The current schema for this input view.
     */
    void pullDownstreamInputSchema(SchemaProvider schemaProvider, final String name, ObjectSchema
            downstreamSchema, String view, ObjectSchema inputSchema);

    /**
     * Snaps should implement this method to propagate schema from upstream snaps to this
     * snap's output views.
     *
     * @param schemaProvider
     * @param name The name of the input view that is connected to an upstream snap.
     * @param upstreamSchema The schema of the upstream output view.
     * @param view The name of this snap's output view that is being modified.
     * @param outputSchema The current schema for this output view.
     */
    void propagateUpstreamOutputSchema(SchemaProvider schemaProvider, final String name,
            ObjectSchema upstreamSchema, String view, ObjectSchema outputSchema);
}

[rpatrick00]

Thanks @ptaylor, this is very helpful!

Now, the next problem is how do I add a “Mapping Root” property to my snap that behaves like the one in the Mapper?

Obviously, I know how to add a property and get its configuration but I am not sure how to get the field to display the inbound schema that was propagated from the previous node (the Mapper) when the user clicks inside the box? Currently, I only get the list of built-in Javascript functions…

Thanks,
Robert

[ptaylor]

Try adding this to your property definition:

.expression(SnapProperty.DecoratorType.ACCEPTS_SCHEMA)