SQL Joins

-
Image
Joins are used to combine rows from two or more tables based on related columns between them. Joins allow you to retrieve data from multiple tables simultaneously, enabling you to create complex queries that fetch data from different sources. There are different types of joins in SQL, including: INNER JOIN Returns only the rows that have matching values in both tables based on the specified join condition. It discards non-matching rows from both tables. Example:           create table t1(x int); insert into t1 values(1); insert into t1 values(1); insert into t1 values(0); create table t2(y int); insert into t2 values(0); insert into t2 values(1); insert into t2 values(1);           select * from t1 inner join t2 on t1.x = t2.y Output: 2. LEFT JOIN (or) LEFT OUTER JOIN Returns all the rows from the left (or first) table and the matching rows from the rig...
Post a Comment

Airflow dynamic dag creation and chain sequentially

-


DAG1:

 import airflow.utils.dates
import airflow.utils.helpers
from airflow import DAG
from airflow.operators.dagrun_operator import TriggerDagRunOperator
from airflow.operators.python_operator import PythonOperator

from airflow.operators.sensors import ExternalTaskSensor

from airflow.models.variable import Variable
from airflow.utils.state import State

from datetime import datetime, timedelta
import dateutil.parser

dag = DAG(
    dag_id="dag1",
    default_args={"owner": "airflow", "start_date": airflow.utils.dates.days_ago(2)},
    schedule_interval="@once", 
    catchup=False
)

def trigger_dag_with_context(context, dag_run_obj):   
    dag_run_obj.payload = {'job': context['params']['job']}
    return dag_run_obj

l1 = []
l2 = []
i = 0
for job in ["job1", "job2"]: 
    trigger = TriggerDagRunOperator(
        task_id= job + "_dag",
        trigger_dag_id="dag2",
        python_callable=trigger_dag_with_context,
        params={"job":job},     
        dag=dag,
        provide_context=True,
        execution_date= '{{execution_date.replace(second=0, microsecond=0) +  macros.timedelta(minutes='+ str(i) +')}}'
    )

    external_task = ExternalTaskSensor(
        task_id= job + '_sensor',
        external_dag_id='dag2',     
        external_task_id = None,
        dag=dag,     
        allowed_states=['success'],       
        execution_delta=timedelta(minutes=-i),     
        poke_interval=1           
    )

    i = i + 1
    l1.append(trigger)
    l2.append(external_task)

zip_ops = list(zip(l1, l2))
flat_ops = [op for tuple in zip_ops for op in tuple]

airflow.utils.helpers.chain(*flat_ops)


DAG2

 import airflow.utils.dates
from airflow.models import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.python_operator import PythonOperator

from helper import *

with DAG(
    dag_id="dag2",
    default_args={"start_date": airflow.utils.dates.days_ago(2), "owner": "Airflow"},
    schedule_interval=None, 
    catchup=False
    ) as dag:

    def run_this_func(**kwargs):
        """
        Print the payload "message" passed to the DagRun conf attribute.
        :param context: The execution context
        :type context: dict
        """ 
        print(kwargs) 
        print(kwargs["dag_run"].conf)
     
     
    run_this = PythonOperator(
        task_id="run_this",
        python_callable=run_this_func,
        provide_context=True,
        dag=dag
    )

    bash_this = BashOperator(
        task_id="bash_this",
        bash_command='echo "Bash Done"'
    )


    run_this >> bash_this

Comments

Post a Comment

Popular posts from this blog

Machine Learning Foundations - Deep Learning Summary - Quiz

-
1) Which of the following statements are true? (Check all that apply) a) Linear classifiers are never useful, because they cannot represent XOR. b) Linear classifiers are useful, because, with enough data, they can represent anything. c) Having good non-linear features can allow us to learn very accurate linear classifiers. d) none of the above 2) A simple linear classifier can represent which of the following functions? (Check all that apply) Hint: If you are stuck, see https://www.coursera.org/learn/ml-foundations/module/nqC1t/discussions/AAIUurrtEeWGphLhfbPAyQ a) x1 OR x2 OR NOT x3 b) x1 AND x2 AND NOT x3 c) x1 OR (x2 AND NOT x3) d) none of the above 3) Which of the the following neural networks can represent the following function? Select all that apply. (x1 AND x2) OR (NOT x1 AND NOT x2) Hint: If you are stuck, see https://www.coursera.org/learn/ml-foundations/module/nqC1t/discussions/AAIUurrtEeWGphLhfbPAyQ d 4) Which of the following statements is true? (Che...
Read more

Machine Learning Foundation - Deep Learning - Programming Assignment

-
1) What’s the least common category in the training data? a) bird b) dog c) cat d) automobile 2) Of the images below, which is the nearest ‘cat’ labeled image in the training data to the the first image in the test data (image_test[0:1])? f 3) Of the images below, which is the nearest ‘dog’ labeled image in the training data to the the first image in the test data (image_test[0:1])? d 4) For the first image in the test data, in what range is the mean distance between this image and its 5 nearest neighbors that were labeled ‘cat’ in the training data? a) 33 to 35 b) 35 to 37 c) 37 to 39 d) 39 to 41 e) Above 41 5) For the first image in the test data, in what range is the mean distance between this image and its 5 nearest neighbors that were labeled ‘dog’ in the training data? a) 33 to 35 b) 35 to 37 c) 37 to 39 d) 39 to 41 e) Above 41 6) On average, is the first image in the test data closer to its 5 nearest neighbors in the ‘cat’ data or in the ...
Read more

SQL Access WebApi with Basic Authentication

-
exec CALLWEBSERVICE 'username','password' CREATE PROCEDURE CALLWEBSERVICE(@Username NVARCHAR(50), @Password NVARCHAR(50)) AS     BEGIN     Declare @Object as Int;     DECLARE @authHeader VARCHAR(8000);        DECLARE @contentType VARCHAR(8000);     DECLARE @postData NVARCHAR(4000);     Declare @json as table(response nvarchar(max))     SET @postData = '{"method":"GetUserVehicles","userid":"1323","companyid":"0"}';        SET @contentType = 'application/json';     SET @authHeader = @Username + ':' + @Password;         SELECT         @authHeader = 'BASIC ' + CAST(N'' AS XML).value(               'xs:base64Binary(xs:hexBinary(sql:column("bin")))'             , 'VARCHAR(MAX)'...
Read more